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(g) Spatially scalable video coding facilitating the derivation of variable-resolution ImagM. 



^ An adaptive technique for encoding and de- 
coding which facilitates the transmission, re- 
ception, storage, or retrieval of a scalable video 
signal. The invention allows this scaling to be 
performed entirely in the spatial domain. In a 
specmc embodiment of the invention this scal- 
ing is realized by adaptively encoding a video 
signal based upon a seleclton token from 
among a multiplicity of predictions from previ- 
ously decoded images, and a selection of com- 
patible sections obtained from up-sampling 
lower resolution decoded images of the cunent 
temporal reference. A technical advantage of 
the invention is that both the syntax and signal 
multiplexing stmcture of at least one encoded 
lower-resolution scate of video is compatible 
with the MPE(5-1 standards. 
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Technical Field 

The invention relates to the encoding of video sig- 
nals, and more particularly, encoding a video signal 
In a manner which pernnits images having a wide 
range of resolutions to be derived from a single en- 
coded signal. 

Background of the Invention 

Worldwide efforts are underway to improve the 
quality of video signal production, transmission, and 
reproduction t>ecause a great deal of commercial im- 
portance is being predicted for improved quality video 
systems. These efforts involve, at least In part. In- 
creasing the resolution with which images are con- 
verted Into representative electrical signals by in- 
creasing the spatial and temporal sampling rates that 
are used to convert video Images into electrical sig- 
nals. This increase in resolution consequently means 
that more data about images must be produced, proc- 
essed, and transmitted in a given interval. 

Video images such as those images In the field 

of view of a television camera are scanned at a pre- 
determined rate and converted into a series of elec- 
trical signals, each electrical signal representing a 
characteristic of a predetermined region of the image 
generally referred to as a picture element (*pel*), or 
pixel. A plurality of the pels taicen together at a pre- 
determined instant of time form what amounts to a still 
picture (Ke„ a frame) representing the nature of the 
image at the predetermined instant of time. Increas- 
ing the quality of video signals produced In this man- 
ner involves, at least in part, the use of a larger num- 
ber of smaller-size pels to represent a given image, 
and the production of a large number of images per 
unit of time. 

As the number of pels for each video image and 
the rate at which images are produced increases, 
there is an increasing amount of video data which 
must be produced, transmitted, and received in a giv- 
en interval. A number of data compression schemes 
have been proposed which attempt to transmit higher 
quality video images using the same numbers of bits 
and the same bit rates used for lower quality images. 
The Motion Picture Experts Group Phase 1 ("MPEG- 
1") standard provides a particular syntax and decod- 
ing process for one such scheme. This standard is set 
forth in International Standards Organization ("ISO") 
Committee Draft 11172-2, "Coding of Moving Pictures 
and Associated Audio for Digital Storage Media at up 
to about 1.5 Mbit/s*, November 1991. 

It may be desirable to obtain one or more lower 
resolution images from a single transmitted high- 
resolution video signal. For example, a video signal si- 
multaneously transmitted to both high- definition tel- 
evision ("HDT\0 standard television receivers 
may have to provide images having a very high de- 



gree of resolution to the HDTV receivers, and images 
having a lesser degree of resolution to the standaid 
receivers. Similarly, the degree of image resolution 
which need be obtained from a video signal displayed 
on a windowed computer screen must t>e varied with 
the size of a particular window in which it is displayed. 

One known method of providing a video signal 
from which images of varying resolution may be de- 
rived is to simultaneously transmit a set of indepen- 
dent replicas of a video sequence; each replica t>eing 
scaled for reproduction at a different level of resolu- 
tion. This approach, referred to as "simulcasting*, is 
simple, but requires an increased bandwidth to ac- 
commodate the transmission of nrnjltlpie independent 
video images. A more bandwidth efficient alternative 
to simulcasting, is scalable video. Scalable video is a 
technique wherein a video signal is encoded and the 
resulting bit-sequence is partitioned so that a range 
of resolution levels may be derived from it depending 
upon the particular signal decoding scheme em- 
ployed at a receiver. 

Unfortunately, the encoding of scalable video Is 
hot provided for v^thin the constraints of most video 
standards. A particular limitation of the MPEG-1 stan- 
dard coding is its lade of provisiorts facilitating scal- 
able video encoding. 

Summary of the invention 

The aforementioned problems are solved, in ac- 
cordance with the principles of the invention, by em- 
ploying a unique, adaptive video encoding and decod- 
ing technique which facilitates the transmissiont r<s- 
ception, storage, or retrieval of a scalable video sig- 
nal. The invention allows this scaling to fc>e performed 
entirely in the spatial domain. In a specific embodi- 
ment of the invention this scaling is realized t>y adap- 
tively encoding a video signal based upon a selection 
taken from among a multiplicity of predk:tions from 
previously decoded images, and a selection of cono- 
patible predictions obtained from up-sampling lower 
resolution decoded images of the current temporal 
reference. 

A technical advantage of the invention is that 
both the syntax and signal multiplexing structure of at 
least one encoded lower- resolution scale of video is 
compatible with the MPE6-1 standards. 

Brief Description of the Drawing 
50 

in the drawing: 

FIG. 1 shows, in simplified block diagram form, 
an Illustration of a video signal encoding/decod- 
ing system which facilitates the field encoding 
65 and decoding of a three-layer spatially scalak>le 

digital video signal in accordance with one exam- 
ple of this inventbn; 

FIG. 2A is an illustration of a group-of-picture 
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structure showing MPEG-1 type predictions 
which may be effected for odd f ield macroblocks; 
FIG. 2B in an Blustration of a group-of- picture 
structure showing single-field predictions which 
may be effected even-field macroblod^s in accor- 6 
dance with one example of this invention; 
FIG. 2C in an Illustration of a group-of- picture 
structure showing dual-field predictions which 
may be effected even-f ield macroblocks in accor- 
dance with one example of this invention; io 
FIG. 3 is a pictorial representation of the slice 
structure employed in the three-layer spatially 
scalable digital video encoding performed in ac- 
cordance with the example of the invention Illu- 
strated In FIG. 1; . 
FIGS. 4A, 4B and 4C show, in simplified block di- 
agram form, the internal architecture of a picture 
encoder which facflitates the encoding of a three- 
layer spatially scalable digital video signal in ac- 
cordance with the example of the invention Blu- 20 
strated in FIG. 1; 

FIG. 4D illustrates the manner In which FIGS. 4A, 
-4B and 4C may-be matedtoobtain a full simplif ied ~ 
block diagram of the internal architecture of a pic- 
ture encoder which facilitates the encoding of a 26 
three-layer scalable digital video signal in accor- 
dance with the example of the invention illustrat- 
ed in FIG. 1; 

FIG. 5 shows, in simplified block diagram form, a 
mode controller for the encoder of FIGS. 4A, 48 30 
and 4C; 

FIG. 6A shows, in simplified block diagram form, 
the basic architecture of an analyzer used within 
the picture encoder of FIGS. 4A, 4B and 4C; 
FIG. 6B is a table showing the reference, input, 36 
bypass control, and output signals for the analyz- 
er of FIG. 6A; 

FIG. 7 shows, in simplified block diagram form, a 
read/write controller for the encoder of FIGS. 4A, 
4Band4C: ^ 
FIG. 8 shows, in simplified block diagram form, 
the internal architecture of a picture store within 
the encoder of FIGS. 4A, 4B and 4C; 
FIG. 9 shows, in simplified block diagram form, 
the internal architecture of a picture decoder 45 
whidi facilitates the decoding of a Source Inter- 
mediate Format ("SIF") odd digftal video signal in 
accordance with the example of the invention I- 
iustrated in FIG. 1; 

FIG. 10 shows, in simplified block diagram form, 50 
the internal architecture of a picture decoder 
which facilitates the decoding of SIF odd and SIF 
even digital video signals in accordance with the 
example of the invention illustrated in FIG. 1; 
FIG. 11 shows, in simplified block diagram form, 55 
the internal architecture of a picture decoder 
which facilitates the decoding of SIF odd, SIF 
even, CCIR-601 odd, and CCIR-601 even digital 



video signals in accordance with the example of 
the invention illustrated in FIG. 1; 
FIG. 12 shows, in simplif ied block diagram form, 
an illustration of a video signal encoding/decod- 
ing system which facilitates the field encoding 
and decoding of a two-layer spatially scalable 
digital video signal in accordance with one exam- 
ple of this invention; 

FIG. 13 is a pictorial representation of the slice 
structure employed in the two-layer spatially 
scalable digital video encoding performed In ac- 
cordance with the example of the Invention Illu- 
strated In FIG. 12; 

FIG. 14 shovws, in simplified block diagram form, 
an illustratton of a video signal encoding/decod- 
ing system which facilitates the frame encoding 
and decoding of a two-layer spatially scalable 
digital video signal in accordance with one exani- 
ple of this Inventton; 

FIG. 15 Is a pictorial representation of the slice 
structure employed In the two-layar spatially 
scalable digital video encoding performed In ao- 
-cordance wlth-the^xample of the Invention lUur 
strated In FIG. 14; 

FIGS. 1 6A, 1 6B and 1 6C show. In simplified block 
diagram form, the Internal architecture of a plc^ 
ture encoder which facilitates the encoding of a 
two-layer spatially scalable digital vWeo signal In 
accordance with the example of t tie invcntton B- 
lustrated in FIG. 14; 

FIG. 16D illustrates the manner in which FIGS. 
16A, 16B and 16C may be mated to obtain a full 
simplified block diagram of the internal architec- 
ture of a picture encoder whfch facilitates the en- 
coding of a two-layer spatially scalable digital vW- 
eo signal in accordance with the example of ther 
invention illustrated in FIG. 14; 
FIG. 17 shov^^, in simplified block diagram form, 
a mode controller for the encoder of FIGS. ISA. 
16B and 1GC; 

FIG. 18 shows, in simplified block diagram form, 
a read/write controller for the encoder of FIGS. 
16A, 16B and 16C; 

FIG. 19 shows, in simplified block diagram form, 
the internal architecture of a picture store within 
the encoder of FIGS. 16A, 16B and 16C; 
FIG. 20 shows, in simplified block diagram form, 
the internal archHecture of a picture decoder 
which facilitates the decoding of a Half Horizontal 
Resolution ("HHR*) digital video signal In accor- 
dance with the example of the invention illustrat- 
ed in FIG. 14; 

FIG. 21 shows. In simplified block diagram form, 
the internal architecture of a picture decoder 
which facilitates the decoding of HHR and CGIR- 
601 digital video signals in accordance with the 
example of the invention illustrated in FIG. 14; 
FIG. 22 shows, in simplified block diagram form. 
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an iflustration of a video signal encoding/decod- 
ing system which facilitates frame- based CCIR- 
601 and field based SIF encoding and decoding 
of a three-layer spatially scalable digital video 
signal in accordance with one example of this in- 6 
vention; 

FIG. 23 is a pictorial representation of the slice 
structure employed in the three-layer spatially 
scalable digital video encoding performed in ac- 
cordance with the example of the invention Hlu- io 
strated in FIG. 22; 

FIGS. 24 A, 24 B and 24C shows. In simplified 
block diagram form, the internal architecture of a 
picture encoder which facilitates the encoding of 
a three-layer spatially scalable digital video sig- is 
nal in accordance with the example of the inven- 
tion illustrated in FIG. 22; 

FIG. 24D illustrates the manner in which FIGS. 
24A, 24B and 24C may be mated to obtain a full 
simplified block diagram of the internal architec- 20 
ture of a picture encoder which facilitates the en- 
coding of a_three-Jayer spatially scalable digital 
video signal in accordance with the example of 
the invention illustrated in FIG. 22; 
FIG. 25 shows, in simplified block diagram form, 25 
a mode controller for the encoder of FIGS. 24A, 
24B and 24C; 

FIG. 26 shows, in simplified block diagram form, 
a read/write controller for the encoder of FIGS. 
24A, 248 and 24C; 30 
FIG. 27 shows, in simplified block diagram form, 
the internal architecture of a picture store within 
the encoder of FIGS. 24A, 24B and 24C; and 
FIG. 28 shows, in simplified block diagram form, 
the internal architecture of a picture decoder 36 
which facilitates the decoding of CClR-601, SIF 
odd, and SIF even digital video signals in accor- 
dance with the example of the inventk>n illustrat- 
ed in FIGS. 24A, 248 and 24C. 

40 

Detailed Description 

FIG. 1 is a simplified block diagram showing an 
illustratton of a video signal encoding/decoding sys- 
tem, in accordance with one example of this inven- 45 
tion. which facilitates the field encoding and decoding 
of a digital video signal having three layers of spatial 
resolution. Signal encoding is performed within func- 
tional block 100, and signal decoding is performed 
within functional block 101. Layer 1 consists of so 
MPEG-1 encoded SIF odd fields and produces an 
MPEG-1 compatible bitstream. Layer 2 consists of 
SIF even fields encoded using adaptive dual/single 
field motion compensated predicttons. Layer 3 con- 
sists of CCIR-601 field-structured pictures which are 55 
encoded using an adaptive choice of temporal predic- 
tion from previous coded pictures, and a spatial pre- 
diction corresponding to a current temporal reference 



obtained by interpolating decoded layers 1 and 2. As 
in the encoding of layer 2, layer 3 is encoded using 
adaptive dual/single field motion compensated pre- 
dictions (the spatially interpolated layer 1 and 2 im- 
ages may be used as spatial prediction for CCIR-601 
fields of corresponding parity). 

As shown in FIG. 1, CCiR-601 digital video sig- 
nals are input to functional block 100 via line 1 02. The 
digital video input signals are spatial and temporal 
samples of a video image and may t>e produced by 
scanning an image field and producing an electrk:al 
signal relating to the characteristics of the image f iekj 
at predetermined points. The characteristics deter- 
mined in the scanning operation are converted into 
electrical signals and digitized. The video input sig- 
nals comprise a succession of digital words, each rep- 
resenting some information at a particular instant of 
time for a small region of the image f ield generally re- 
ferred to as a pel. A complete set of digital represen- 
tations for the image at a particular instant of time is 
called a frame. Each frame may be consMered to be 
composed of two interlaced fields representing odd- 
and even- numbered horizontal lines In the image. 

The digital video signal encoded and decoded by 
this invention may be a monochrome vkieo signal or 
a color video signal. In the case of a monochrome vid- 
eo signal, each frame may comprise a set of digital 
representations ofthe brightnessorintensHy of a two- 
dimensional array of pels which make up a video Im- 
age. In the case of a color video signal, each picture 
comprises not only a brightness component but also 
a color component For example, in the CCIR-601 
4:2:2 recommendation, a color video signal picture 
(i.e., a temporal sample of the image) may be com- 
posed of a luminance frame, Y, of 720 horizontal pels 
X 480 lines and two chrominance frames, Cb and Cr, 
at 1/2 resolution of 360 horizontal pels x 480 lines 
each. A sequence of such pictures may t>e transmitted 
at a rate of 29.97 pictures per second. The luminance 
or chrominance frame is formed as the interlaced uni- 
on of the two constituent CCIR-601 fields, while 
CCIR-601 4:2:0 frames can be derived by f Itering and 
sub-sampling the respective 4:2:2 CCIR-601 chromi- 
nance frames. 4:2:2 is a standard sampling structure 
for CCIR-601 video having a 4 to 2 to 2 sampling ratto 
of Y, Cb, and Cr, respectively. For the purpose of il- 
lustrating a specific example of the invention, the de- 
scription below assumes that the digital vkJeo signal 
received via line 102 is a video signal in accordance 
with the CCIR-601 4:2:0 resolution. Those skilled In 
the art will appreciate that the principles of the inven- 
tion are applicable to other types of video signals, 
such as HDTV video signals. 

To assist in the description of the example ofthe 
invention shown in FIG. 1, some terminology shoukt 
be defined. A typical block is an 8- horizontal- row t>y 
8-vertical-column array of contiguous pels. Blocks 
may be groups of luminance data or groups of chro- 
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minance data. A typical macroblock is composed of 
four contiguous 8x8 luminance data blocks and the 
two 8x8 chrominance data blocks corresponding to 
the area of the image represented by the four luml- 
nance data blocks. A slice Is one horizontal row of 
macroblocks starting at the left edge of the picture 
and ending at the right edge of the picture. A lumi- 
nance frame is formed as an interlaced union of two 
CCIR-601 luminance fields. One field comprises 
even-numbered horizontal rows of pels and the other 
field comprises odd-numbered horizontal rows of 
pels. 

In the example of the invention shown in FIG. 1 . 
a plurality of picture types are encoded and decoded. 
Specifically, l-pictures, P-pictures. and B-pictures are 
encoded and decoded, l-pictures or intra-coded pic- 
tures are pictures which are encoded and decoded 
without reference to any other pictures (I.e.. no tem- 
poral prediction is required). P^pictures, or predicted 
pictures are encoded in light of a previous pictures 
(namely In light of a temporal prediction from previ- 
ously decoded I- or P- pictures). Motion compensa- 
_t]pjLmay be usedtp_mpdiJ^P^pict^^^^ or 
bidirectionally predicted pictures are pictures which 
are encoded in light of characteristics of previously 
decoded I- or P-piclures and future decoded P- or I- 
pictures. As in the case of P-pictures, B-piclures may 
also be encoded by using motton compensatton. In 
appropriate circumstances. P-pictures and B-pictures 
may have some of their blocks encoded in the same 
fashion that the blocks of the l-pictures are encoded. 
Le., without reference to other pictures ("intra cod- 
ing"). 

A Group-of-Pictures (GOP) structure illustrating 
MPEG-1 type prediction options for macroblocks of 
odd fields only is shov^ in FIG. 2A. The even fields 
are drawn for completeness and are not used for pre- 
diction. An l-picture (1 0) field does not require tempo- 
ral prediction. P-picture fields require temporal pre- 
diction from the previously decoded I- or P- picture 
fields. In this example. PS Is predated from 10. B-pic- 
tures are bidirectionally predicted from the previous I- 
or P- picture fields and the next P- or I- picture field. 
Both B2 and B4 are bidirectionally predicted from 10 
and PS. In the coding/decoding systems to be descri- 
bed. MPEG-1 predictton Is applied on the SIF odd 
field macroblocks to provide MPEG-1 compatibility. 

FIG. 2B shows a GOP structure of single field 
prediction options for macroblocks of even fields. P- 
picture macroblocks are predated from either the im- 
mediate previously decoded odd field, or the immedi- 
ate previously decoded even field, which belong to 
coded I- or P- picture fields (whichever gives a better 
prediction). In this example, macroblocks of PI are 
predicted from 10 only, as 10 is the only field available 
for prediction. P7 is predicted from either P1 or P6 on 
a macroblock basis, whichever gives a better esti- 
mate. B-picture macroblocks are predicted bidirec- 



tionally from decoded I- or P- fields. In this example, 
macroblocks of B3 reference either 10 or PI for for- 
ward prediction, and reference either P6 or P7 f or 
backward prediction. 
6 The GOP structure of FIG. 2C shows dual fieW 

prediction options for macroblocks of even fields. P- 
picture macroblocks reference both the immediate 
previously decoded odd and even f ields. The predic- 
tion used is the average of the predictions obtained 
10 from both of these referenced fields. In this example, 
P7 macroblocks reference both PI and P6, and use 
the average of the predictions made from these ref- 
erenced fields. B-picture macroblocks arc predicted 
either in the forward direction or in the backward dl- 
16 rection, but not both. They reference either the Im- 
mediate previously decoded odd and even fields, or 
the immediate future decoded odd and even fields, 
which belong to coded I- or P- pteture fields. Regard- 
less of the direction, the prediction Is the average of 
20 best prediction nr^croblocks of the tv« referenced 
fields. 

As shown In FIG. 1, the digital video signal re- 
ceived on lineal 02 Is decimated to SIF vWeo by SIF 
dedrnatOT T63 for the encoding layem 1 and 2. 
26 Such decimators are well-known In the art The deci- 
mated video Is output by decimator 103 to switch 104, 
which directs odd numbered horizontal rows (or slic- 
es) to line 1 05, and even numbered horizontal rows to 
line 106. The odd horizontal rows comprise an odd 
30 video field, the even horizontal rows comprise an 
even video field. The SIF odd fields are directed to 
MPEG-1 SIF encoder 107, where they art encoded 
and sent as a bitstream to video multiplexer 108 (this 
encoded bitstream corresponds to the first video lay- 
36 er). MPEG-1 SIF 107 encoder also outputs a decoded 
SIF odd signal to even field SIF encoder 109. and 
switch 110 via lines 111 and 112, respectively. Even 
field SIF encoder 109 encodes the SIF even fields 
into a bitstream (corresponding to the second video 
40 layer) which is output to video multiplexer 108. Even 
field SIF encoder 109 also outputs a decoded SIF 
even signal to switch 110 via line 113. Switch 110 al- 
lows either a decoded SIF odd field, or a decoded SIF 
even field to be input to horizontal interpolator 114 (a 
46 decoded SIF odd field is selected if an odd CCIR-601 
field is being encoded, and a decoded SIF even f ieW 
is selected If an even CCIR-601 field is being encod- 
ed). Horizontal interpolator 114 up-samples the odd 
and even fields received from encoders 107 and 109 
50 to CCIR-601 resolution. This up-sampled signal Is 
output via line 115 to field-structure CCIR-601 encod- 
er 116, where it is used for obtaining a spatial predic- 
tion of the third video layer. Field-structure CCIR-601 
encoder 114 encodes the CCIR-601 fields of the dlg- 
55 ital video signal received via line 102. with an adap- 
tive selection between temporal and spatial picture 
predictions. The resulting encoded bitsUeam (corre- 
sponding to the third video layer) is output by field- 
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structure CCIR-601 encoder 116 to video multiplexer 
108. 

Video multiplexer 108 multiplexes the encoded 
bitstreams into a single bitstream. This single bit- 
stream is transmitted to video demultiplexer 117 
where it is demultiplexed into individual bitstreams, 
one for each of the video layers. MPEG-1 SIF decoder 
118 decodes the bitstream corresponding to the first 
video layer, thereby reconstituting an SIF odd field 
video signal which is output on fines 119, 120, and 
121. Even field SIF decoder 122 decodes the bit- 
stream corresponding to the second video layer (errv 
ploying an optional prediction based upon the SIF odd 
fields decoded by MPEG-1 SIF decoder 118). This 
yields an SIF even field video signal which is output 
on lines 123 and 124. Switch 125 allows either a de- 
coded SIF odd field, or a decoded SIF even field to 
be output on line 126 (depending upon which type of 
field is being decoded within functional block 101). 
Switch 127 allows either a decoded SIF odd field, or 
a decoded SIF even field to be input to horizontal in- 
ter poiator 1 28 (a dec oded SIF odd field is selected If 
an odd~CCIR-601 field Is belng~decoded, and a de- 
coded SIF even field is selected if an even CCiR-601 
field is being decoded). Horizontal interpolator 128 
up-samples the odd and even fields received from de- 
coders 118 and 122 to CCIR-601 resolution. Field- 
structured CCIR-601 decoder 129 decodes the bit- 
stream corresponding to the third video layer (errv 
ploying an optional spatial prediction based upon the 
interpolated decoded SIF odd and even fields from 
layers one and two) to reconstitute a CCIR-601 video 
signal v^ich is output on line 130. 

A representation of the slice structure employed 
in the encoding performed within functional block 100 
of FIG. 1 is shown in FIG. 3. The odd and even fields 
are encoded one after the other. Each SIF odd slice 
block may be used in the optional spatial prediction of 
blocks of CCIR-601 odd slice corresponding to the 
same spatial image. Similarly, SIF even slice blocks 
may be used in the optional spatial prediction of 
blocks of CCIR-601 even slices. The slices of the SIF 
odd, SIF even, CCIR-601 odd and CCIR-601 even 
fields must be multiplexed as shown for the images 
to be decoded correctly. 

Although FIG. 1 shows three separate encoders 
being employed to accomplish the encoding of the 
three video layers, in a particular embodiment of the 
invention, the encoding for all three layers may be ac- 
complished sequentially within a single adaptive en- 
coder. FIGS. 4A, 4B and 4C show a simplified block 
diagram of the internal architecture of one example of 
such an adaptive picture encoder. As this encoder 
must perform three distinctly different types of en- 
coding, certain analyzers and sut>-systems within it 
must be enabled and disabled as a function of the 
particular type of encoding being performed at a giv- 
en time. This enabling/disabling is performed as func- 



tion of control signals generated by a mode controller. 
Mode controller 500, shown in FIG. 5, generates the 
signals to control the mode of operation for the encod- 
er of FIGS. 4A, 4B and 4C. Mode controller 500 re- 

6 ceives input signals from an independently running 
processor (not shown in FIG. 6) which generates the 
input signals to cause a particular slice arrangement 
to be realized according to pre-progranrvned Informa- 
tion and the type of picture being processed. The in- 
to put signals provided to mode controller 500 indude: 
picture_structure, which identifies whether a picture 
is frame-structured or field-structured; picture_type, 
which identifies an I-, P-, or B- picture; 
slice_mux_order, which identifies the slice multiplex- 

15 ing structure employed; and slice_number, which pro- 
vides a count of the slice being processed. The output 
of mode controller 500 provides bi-level fO* or "I") 
control signals including: srf.slice fl" when the current 
slice is from an SIF resolution image); slf_odd_slioe ri* 

20 when the current slice is from an SIF odd field); 
ccir.slice (*1" when the current slice is of CCIR-601 
resolutk)n); ccir^compat.slice fl* when current slice 
is of CCIR-601 resolution and uses a compatible pre- 
diction); and bypass control signals dis_inter, 

25 dis_spatial, and dis_dual (which disat)le the inter/in- 
tra, spatial/temporal prediction, and dual/single f iekS 
predictk>n analyzers). 

The inter/intra, spatial Aemporal prediction, and 
dual/single field motk>n compensated prediction 

30 ("dual/single field MCP") analyzers employed within 
t he encoder of FIGS. 4A and 4B (as well as those ana- 
lyzers employed within other encoders and decoders 
which will be discussed below) are all of the same ba- 
sic configuration as analyzer 600 illustrated in FIG. 

36 6A. Analyzer 600 accepts as input a reference signal 
on line 601, a first input signal ("Input 1") on line 602. 
a second input signal (""Input 2") on line 603, and a by- 
pass signal on line 604. The reference signal is al- 
ways the original video signal input to the particular 

40 encoder in which the analyzer is located. A signal in- 
dicative of the difference between the reference sig- 
nal and the Input 1 signal is output by summing ele- 
ment 605. Similarly, a signal indicative of the differ- 
ence between the reference signal and the Input 2 

45 signal is output by differencing element 606. The en- 
ergy in the signals output by differencing elements 
505 and 606 is computed, respectively, by summing 
elements 607 and 608. This energy summation may 
be arrived at by any suitable method using either a 

so sum of squares or sum of absolute value criteria. The 
energy summations are then fed to comparator 609. 
If the input from summing element 607 is smaller or 
equal to the input from summing element 608, the out- 
put of comparator 609 is "I*. If the Input from summing 

55 element 607 is greater than the input from summing 
element 608, the output of comparator 609 is "0". 
However, if the bypass signal input on line 604 is set 
to '1*, the comparator output is bypassed, and the 
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output of analyzer 600 is forced to "Q-. FIG. 6B is a ta- 
ble showing the various reference. Input 1. Input 2, 
bypass control, and output signals for the various 
types of analyzers employed within the encoders dis- 
cussed below. 

In performing the MPEG-1 SIF encoding (layer 
1). the SIF odd field macroblocks output by SIF dec- 
irnator 103 (FIG. 1) are received by the adaptive pic- 
ture encoder of FIGS. 4A. 4B and 4C via line 401 (FIG. 
4A). As MPEG-1 encoding does not use spatial or 
dual field prediction, mode controller 600 sets both 
the dis^spalial and dis^dual signals to '1". This effec- 
tively disables spatialAemporal prediction analyzer 
402. and dual/single field MCP analyzer 403. result- 
ing in the compatibHrty signal fsJ-Comp-) being set 
to "0", and the dual/single field motion compensation 
signal (-dual/single_f ield_mc") being set to -0^ In ad- 
dition, the sif.slice and sif.odd.slice signals are set 
to "1^ and the ccir_compat_slice signal is set to t)" 
by mode controller 500. Respectively, these signals 
cause switch 404 to dose and switch 405 to open. 

II the current picture Is an l-picture, mode control- 
ler 500 oiiipuls.a dis Jnter bypass control sig nal of "1 
thereby disabling inter/lntra analyzer 406 so that a 
outputs a -0° inler/intra signal. This causes switch 407 
to remain open so that dif ferencer 408 receives a pre- 
diction of zero at its negative input. As a result, the in- 
put video signal on line 401 is passed, unaltered, 
through differencer 408 to discrete cosine transform 
circuit (-DCr-) 409. If the current picture is not an I- 
picture. inter/intra analyzer 406 is not disabled, but 
rather employed to calculate the inter-coding predic- 
tion error and inlra-coding variances. As shown In 
FIG. 6B, and intra-coding variance is calculated by 
computing the square of the difference between the 
original video signal (received on line 401 in FIG. 4A) 
and a signal representing the average energy of the 
original video signal (the average energy is computed 
by average computer 410 in FIG. 4), Inler-coding pre- 
diction error variance is calculated by computing the 
square of the difference between the original video 
signal and an inter prediction signal received via line 
411 (FIG. 4). An Inter/intra signal is generated so as 
to select whichever of these computed variances rep- 
resents the smaller energy. The resulting inter/intra 
signal controls switch 407 and determines what differ- 
encer 408 will subtract from the video signal on line 
401 before the signal is passed to DCT 409. 

OCT 409 converts each 8x8 block into an 8x8 ma- 
trbc of transform coefficients and passes the coeffi- 
cients along to forward quantizer 412. Forward quan- 
tizer 412 quantizes each coefficient based on wheth- 
er the current macroblock is intra or inter coded, arid 
a quantization parameter ("quant- par mtr^. This 
quantization parameter (which controls the coarse- 
ness of the image being encoded) is determined in 
light of the characteristics of the digital input signal 
communicated to quantizer adapter 413 via line 414, 



the fullness of buffer 415. and in light of coding stal- 
i'stics compBed by coding statistics processor 41S. 
The quantized coefficients are forward scanned by 
forward scanner 417 in an predetermined order that 
6 facilitates the encoding of the quantized data by va- 
riable/fixed word-length encoder and overhead mul- 
tiplexer (V/FWL encode^ 418. V/FWL encoder 418 
encodes the quantized data and outputs it as a bH- 
stream to buffer 415. 
fO In addition to transmitting encoded transform 

coefficients. V/FWL encoder 418 also encodes and 
transmits a number of control signals and overhead 
data within the output bitstream, including: the quant- 
parmU signal, the inter/intra signal, nnotion vectors. 
16 the picturejype signal, the sj^comp signal output 
by spatial/Temporal prediction analyzer 402. ths 
dual/singlejield_mc signal, and the reccn_ signals 
(which are employed in the decoding of the bitstream 
and will be discussed later). The V/FWL Encoder also 
20 provides coding statistics information to coding stat- 
istics processor 416. Buffer 413 outputs th® receh/ed 
bitsueam at the appropriate rate for acceptanoa by 

vWeo multiplexer 108 (FjSaJl. , 

Within the adaptive picture encoder of FIGS. 4A, 
25 4B and 4C, the SIF odd slic© Is locally decoded 
through the operations of inverse scanrwssr 419. In- 
verse quantizer 420, and inverse dIscretQ coslrta 
transform circuit ("IDCT') 421, which perforim the In- 
verse functions of forward scannec 417, forwamd 
30 quantizer 412. and DCT 409, respectively. The de- 
coded error signal is added by summing clement 422 
to the prediction signal (evident at the negathfa input 
of differencer 408). The resulting signal Is written to 
next picture store 423 via write.nejct switch 424, artd 
36 to reconstructed SIF one-slice store (°recon slioB 
stored 425 via swHch 404. The signal does not reach 
horizontal interpolator 426 as switch 405 Is open as a 
result of the "0'' value of the cctr_compat_slice signal. 
Before the encoding of an I- or P-picture. the contents 
40 of next picture store 423 are transferred to previous 
picture store 427 via vi^e^prevlous (°write_prev^ 
switch 428. 

The states of write.neKt switch 424 and 
write^rev switch 428 are controlled by read/write 
45 conuoller 700 (FIG. 7). As with mode controller 5CM). 
read/write controller 700 receives input signals from an 
independently running processor (not shown in FIG. 7). 
The input signals provided to read/write controller 7G0 
include: picture_structure; slio8_mux_order, and 
so slice^number (all of which wer© previously dis- 
cussed). The output of read/write controller 700 pro- 
vides bi-level fO'' or °1°) control signals including: 
write^prev and vwHe_next signals, which enable writ- 
ing to the previous and next picture stores (427 and 
55 423 of FIG. 4B); recon_odd_write and recon_ccir_wrl- 
le, which allow the storing of reconstructed odd field 
and CCIR-601 video to the previous and next picture 
stores (FIG, 4B); and recon.odd.read and recon- 
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_ccir_read. which allow odd f ield and CCIR-601 video 
to be read from the previous and next picture stores 
(FIG. 4B). Depending on whether recon_odd_write 
(or recon_odd_read) is enabled or disabled, the use 
of an "odd field" or an "even field" is indicated. All of 5 
these "recon_" signals are encoded into the bitstream 
by V/FWL encoder 418 (FIG. 4C). 

FIG. 8 shows a simplified block diagram of a pic- 
ture store (800), one such store is employed for stor- 
ing a previous picture, and another for storing a next to 
picture within the adaptive picture encoder of FIGS. 
4A, 4B and 4C. As shown, picture store 800 contains 
a separate memory for CCIR-601 odd and even fields 
(801, 802), as well as separate memories for SIF odd 
and even fields (803, 804). Picture store 800 is con- is 
figured to allow the storage and retrieval of pictures 
in response to read/write signals generated by 
read/write controller 700. 

If the current picture being processed by the 
adaptive picture encoder of FIG. 4 is a P- or B- picture, 20 
motion estimator 429 generates motion vectors by 
finding the best match of the odd- field macroblocks 
input on line 401 with the decoded reference odd- 
field images in next and previous picture stores 423 
and 427. The motion vectors are sent to dual/single 25 
field motion compensated predictor ("dual/single field 
MC predictor^ 430 which calculates a prediction of 
the video on line 401 based upon the received motion 
vectors, the contents of next and previous picture 
stores 423 and 427. and the picture_type signal. The 30 
prediction is output by dual/single field MC predictor 
430 on line 431 and passed to switch 407 via switches 
432 and 433 (as this is MPEG-1 encoding. 
dual/single_field_mc and s_t_comp arc set to "0"). 
This completes the MPEG-1 encoding of an SIF odd 35 
slice. 

As is shown in FIG. 3, the next slice to be encod- 
ed is a CCIR-601 odd slice. For encoding a CCIR-801 
layer all modes are available and all analyzers are 
used (Le., all 'dis_* signals generated by mode con- 40 
troller 600 are set to "0^. The ccir_compat_s!ice sig- 
nal generated by mode controller 500 is set to "1 and 
the sif_stice signal generated by mode controller 500 
is set to "0". The macroblocks representing the CCIR* 
801 odd slice are received via line 401. As with pre- 4S 
viously described encoding scheme, signal differenc- 
er408 outputs a prediction error macroblocks. Each 
8x8 block within each prediction error macroblock is 
converted into an 8x8 matrbc of transform coefficients 
bj/^DCT 409. The coefficients are then quantized and 60 
forward scanned by forward quantizer 412 and for- 
ward scanner 417. They are then passed to V/FWL 
encoder 418, buffer 415, and video multiplexer 108 
(FIG. 1), which perform as discussed earlier for 
MPEG-1 SIF odd coding. 55 

In the local decoding loop of the encoder, the out- 
put of forward scanner is inverse scanned, inverse 
quantized and inverse transformed to reconstruct the 



quantized prediction error signal. This decoded error 
signal is added back to the prediction signal (evident 
at the negative input of differencer 408) by summing 
element 422 to generate a reconstructed CCIR-601 
signal. This reconstructed signal is saved in next pic- 
ture store 423. 

The prediction signal for this macroblock was ob- 
tained through the following sequence of operations. 
Motion estimator 429 first compared the input macro- 
blocks of CCIR-601 resolution on line 401 with the de- 
coded reference CCIR-601 Images within the previ- 
ous and next picture stores 427 and 423. If it is an I- 
picture (as indicated by the picture_type signal, no 
motion estimation is performed. Motion estimator 429 
computes two sets of motion vectors, one for dual 
field prediction mode and one for single field predic- 
tion mode. These motion vectors are sent to 
dual/single field MC predictor 430 which, based upon 
these vectors, computes the prediction for t>oth cas- 
es (one being output on line 431 . and the other on line 
434). Dual/single field MOP analyzer 403 compares 
the two predictions and selects the one that gives a 
lower prediction error. A dual/slngle_f ield_nK signal 
is output by dual/single field MOP analyzer 403 80 
that switch 432 is directed to the output of dual/single 
field MC predictor 430 which offers the least predic- 
tion error. Since the ccir_compat_srice signal is set to 
"1 recon slice store 425 outputs the current SIF slice 
to horizontal interpolator 426 (which corresponds to 
horizontal interpolator 109 in FIG. 1). Spatial/temporal 
prediction analyzer 402 compares the Interpolated 
SIF signal (spatial prediction) with the output of 
dual/single field MC predictor 430 (temporal predic- 
tion) to determine which signal gives a better estimate 
of the input video. Spatial/temporal predictk>n analyz- 
er 402 outputs a compatibility signal (s_t_comp) to al- 
low the best choice of spatial or temporal prediction 
to be passed to inter/intra analyzer 406. Inter/intra 
analyzer 406 determines whether intra-coding or in- 
ter-coding is to be used by comparing the variance of 
the signal on line 401 to the variance of the inter pre- 
diction signal computed using the prediction signal re- 
ceived via line 411. An inter/intra signal is generated 
so as to cause switch 407 to allow whichever of these 
signals corresponds to the lower of the two variances 
to be passed to differencer 408. The inter/intra signal 
opens switch 407 if intra-coding is selected or closes 
it if inter predictbn is selected. This completes the 
loop for coding CCIR-601 odd slice. 

The alternate encoding of SIF odd slices arKj 
CCIR-601 odd slices continues until the odd field Is 
complete. Next the encoding of SIF even slices and 
CCIR-601 even slices is performed. The encoding of 
an SIF even slice is similar to that of a CCIR-601 slice 
except that for SIF even slice encoding the 
ccir_compat_slice signal generated by mode control- 
ler 500 is set to "0". and the dis.spatial signal gener- 
ated by mode controller 500 is set to "1*. This causes 
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switch 405 to be opened and spatial/temporal predic- 
tion analyzer 402 to be disabled. 

As shown in FIG. 3. af ler an SIF even slice is en- 
coded, a CCIR- 601 even slice is encoded. This en- 
coding is similar to that of a CCIR-601 odd slice, but 
a reconstructed SIF even slice stored in recon slice 
store 425 is used as a basis for spatial prediction. The 
alternate encoding of SIF even slices and CCIR-601 
even slices continues until the even field is complete. 

As discussed above, during an odd-field, video 
multiplexer 108 (FIG. 1) receives the bilstreams relat- 
ed to the SIF odd, and CCIR-601 odd slices; whereas 
during an even field it receives SIF even, and CCIR- 
601 even slices. Video multiplexer 108 multiplexes 
these received signals and transmits them to video 
demultiplexer 115 (FIG. 1) where they are demulti- 
plexed and passed on to decoders 116, 120 and 124 
(FIG. 1). 

FIG. 9 shows a simplif ied block diagram of the in- 
ternal architecture of a picture decoder which facili- 
tates the decoding of a low-resolution layer digital vid- 
eo signal (MPEG-1) encoded by the encoder of FIGS. 
._4A,^4B„and. 4C.^.A„b]tsU«am cgntaining coefficients 
representing an encoded video field is sent to buffer 
901 from video demultiplexer 116 (FIG. 1). This bit- 
stream also contains the control signals and over- 
head data transmitted by V/F\AfL encoder 418 (FIG. 
40). Buffer 901 passes this signal to slice identifier 
902, which scans the bitstream for an sK_odd_slice 
signal of "1" (indicative of an SIF odd-field slice), and 
in response closes switch 903, and inputs the bit- 
stream to variable/fixed word length decoder and 
overhead demultiplexer CV/FWL decoder") 904. 
V/FWL decoder 904 decodes the control signals and 
overhead data within the bitstream, including: the 
picture_type signal, which is sent to motion comper*- 
sated predictor 905; the motion vectors, which are 
sent to motion compensated predictor 905; the in- 
ter/intra signal, which is sent to switch 906 and to in- 
verse quantizer 907; and the quant-parmtr signal, 
which is sent to inverse quantizer 907. The V/FWL de- 
coder outputs the coefficient related data of the en- 
coded video signal to inverse scanner 908 which re- 
arranges the coefficients and passes them to the In- 
verse quantizer 907. Based upon received control In- 
formation, inverse quantizer 907 reverses the opera- 
tion performed by forward quantizer 412 (FIG. 4A). 
The output of inverse quantizer 907 is inversely trans- 
fer med by IDCT 909 and added, by summing element 
910, to a prediction signal received from switch 906. 
The output of summing element 910 is the recon- 
structed SIF odd slice macroblock. This reconstruct- 
ed macroblock is also stored in next picture store 911 
if it is an I- or a P- picture. Write_next switch 912 Is 
closed If the current field is an I- or P- picture (as de- 
termined from the picture.type signal). Before decod- 
ing of I- or P- picture, the contents of next picture store 
911 are transferred to previous picture store 913upon 



closing of write_prev switch 914. 

Motion compensated predictor 906 computes the 
prediction of the current signal based on the picture_ty- 
pe signal, the received motion vectors, and the con- 
5 tents of next and previous picture stores 911 and 91 3. 
This prediction signal is added to the SIF odd slice 
macroblock passing through sunvr^ing element 910 If 
the current picture being decoded Is inter coded (In- 
ler/intra set to "I*, and switch 906 dosed). However, 
to if t he cur rent picture being decoded is intra coded (In- 
ler/intra set to •0", and switch 906 opened), a signal 
of zero is added to the reconstructed SIF odd slice 
macroblock passing through summing element 910. 
FIG. 10 shows a simplified block diagram of the 
15 internal architecture of a picture decoder which facil- 
itates the decoding of an inter nr>ediate-resolution digital 
video signal encoded by the encoder of FIGS. 4A, 4B 
arwl 4C. This decoder can decode t)Olh SIF odd and SIF 
even fields, and functions in a manner simHar to that of 
20 the decoder of FIG. 9. An incoming bitstream passes 
through buffer 1001 and slice Wentlfler 1002. Slice Wen- 
tifier 1002 scans the bitstream for an 8«Lsltoe_start_co- 
de. Upon finding this start code, slice Identifier 1002 
sets sif_slice signal to "I* (indicative of an SIF odd or 
25 even bitstream), and in response closes switch 1003, 
and inputs the bitstream to V/FWL decoder 1004. 
V/FWL decoder 1 004 decodes the control signals arKi 
overhead data wHhin the bitstreanu including: the 
picture_typc signal, which is sent to dual/single field 
30 MC predictor 1 005; the motion vectors, which are sent 
to switch 1006; the inter/intra signal, which Is sent to 
switch 1007 and inverse quantizer 1008; the 
quan^parmtr signal, which is sent to inverse quantizer 
1008; the duaiysingle_field_mc signal, whk*i controle 
36 swRches 1006 and 1009; and the nBCon_odd_write 
and recon_odd_read signals which are sent to next 
and previous pictures stores 1010 and 1011. For an 
SIF slice macroblock, V/FWL decoder 1004 output 
the coefficient related data of encoded video to In- 
40 verse scanner 1012, inverse quantizer 1008, arKl 
IDCT 1013. The output of IDCT 1013 Is added. Ijy 
summing element 1014, to a prediction signal re- 
ceived from switch 1007. The resulting decoded SIF 
slice macroblock is written into next picture store 1010 
45 via write_next switch 1015 if it is an I- or a P- picture. 
Before decoding of I- or P-picture. the contents of 
next picture store 1010 are transferred to previous 
picture store 1011 via the closing of write_prev switch 
1 01 6. The dual/single_f ield_mc signal directs the mo- 
60 tioh vectors output by V/FWL decoder 1 004 to the ap- 
propriate input of dual/single field MC predictor 1005, 
which computes a prediction for the selected mode. 
If an inter-coded picture is being decoded, switch 
1 007 passes the prediction data to summing element 
55 1 014, where it is added to the output signal of IDCT 
1014. However, if the current picture being decoded 
is intra coded (inter/intra set to "0", and switch 1007 
opened), a signal of zero is added to the reconstituted 
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macroblock passing through summing element 1014. 

FIG. 11 shows a simplified block diagram of the 
internal architecture of a picture decoder which facil- 
itates the decoding of a high-resolution digital video 
signal encoded by the encoder of FIG. 4. This decod- 
er can decode SIF odd, SIP even, and all CCIR-601 
fields. An incoming bitstream passes through buffer 
1101 and slice identifier 1102. Slice identifier 1102 
scans the bitstream for an sif^slice^start code, and 
upon finding it sets the srf_slice signal to "l^ (indica- 
tive of an SIF odd or even slice), the ccir_slice signal 
to "1 " (indicative of a CCIR-601 odd or even slice), and 
the ccir_compat_sllce signal to 'I* (indicative of a 
CCIR-601 slice encoded using a compatible predic- 
tion). The bitstream is then passed to V/FWL decoder 

1103. V/FWL decoder 1103 decodes the control sig- 
nals and overhead data within the bitstream. includ- 
ing: the picture_type signal, which is sent to 
dual/single field MC predictor 1104; the motion vec- 
tors, which are sent to switch 1105; the interAintra sig- 
nal, which is sent to switch 11 06 and Inverse quantizer 
1 107; the quan_parmtr signal, which is sent to inverse 
quantizer -1107; the ^ dual/single_f ield_mc signal, 
which controls switches 1106 and 1108;the s_t_comp 
signal, which controls switch 1109; and the re- 
con_odd_wrile, recon_odd_read, recon_ccir_write. 
and recon_ccir_read signals which are sent to next 
and previous pictures stores 1110 and 1111. 

The decoding of SIF odd and SIF even slices is 
performed this decoder In primarily the same manner 
as in the decoder of FIG. 10. and will not be discussed 
in detaH. However, the decoder of FIG. 11 performs 
one additional step In SIP decoding. When t he current 
slice being decoded is an SIP slice, switch 1112 is 
closed and a reconstructed SIF slice is stored in recon 
slice store 1113. This stored slice will spatially inter- 
polated and used for spatial prediction. 

The decoding of a CCIR-601 slice macroblock 
is performed in response to the detection of a 
ccir_slice_start_code or a ccr_compat_sIice_start_code 
by slk^ identifier 1102. the ccir_slice signal and/or 
the ccir.compat_slice signal is set to "1". In decoding 
a CCIR-601 slice macroblock, the V/FWL decoder 
outputs the coefficient related data of encoded video 
signal to inverse scanner 1114. inverse quantizer 
1107.and IDCT 1116. The output of IDCT 1115 is add- 
ed. by summing element 1116, to a prediction signal 
received from switch 1108. The resulting decoded 
CCIR-601 slice rr^croblock is written into next picture 
store 1110 via write_next switch 1117 if it is an I- or a 
P- picture. Before decoding of I- or P- picture, the con- 
tents of next picture store 1110 are transferred to pre- 
vious picture store 1111 via the closing of write_prev 
switch 1118. The dual/single_field_mc signal directs 
the motion vectors output by V/FWL decoder 1103 to 
the appropriate input of dual/single field MC predictor 

1104. which computes a prediction for the selected 
mode. Switch 1119 is dosed for all CCIR-601 slices 



which employ spatial prediction (indicated by a 
ccir_compat_slice signal of "1*). When switch 1119 is 
closed, the SIF slice in recon slice store 1113 is inter- 
polated by horizontal interpolator 1120 and passed to 

6 switch 1109. Switch 1109 allows either the spatial 
prediction consisting of the interpolated reconstruct- 
ed SIF slice, or the temporal prediction output by 
dual/single field MC predictor 1104 to be used for 
the current CCIR-601 signal (depending upon the 

10 s_t_comp signal decoded by V/FWL decoder 1103). 
If an inter-coded picture is being decoded, switch 
1106 passes the prediction data to summing element 
1116 where it is added to the signal output by IDCT 
1115. However, if the current picture being decoded 

16 is intra coded (inter/intra set to •0*. and switch 1106 
opened), a signal of zero is added to the reconstituted 
macroblock passing through summing element 1116. 

A video signal encoding/decoding system, in ac- 
cordance with a second example of the invention, is 

20 shown in FIG. 12. This system facilitates the field en- 
coding and decoding of a digital video signal having 
two layers of spatial resolutk>n. Signed encoding is 
performed within functional block 12(X), arKl signal 
decoding is performed within functional k>lock 1201. 

25 Layer 1 consists of MPEG-1 encoded SIP odd fields 
and produces an MPEG-1 compatible bitstreanu Lay- 
er 2 consists of CCIR-601 field-structure pictures (ttie 
odd field of which are encoded using an adaptive 
choice of temporal prediction from previous coded 

30 pictures, and a spatial predictton corresponding to a 
current temporal reference obtained by spatialy inter- 
polating decoded layer 1). 

As shown in FIG. 12, CCIR-601 digital video sig- 
nals are input to functional block 1200 via line 1202. 

36 Switch 1203 allows only the odd fields within the in- 
put video signal to be passed to SIP decimator 1204, 
where they are decimated to SIP video for the encod- 
ing of layer 1. Such decimators are well-known in the 
arL SIF decimator 1204 outputs decimated odd fiekl 

40 macroblock slices via line 1205. The SIP odd fields 
are directed to MPEG-1 SIP encoder 1206. where 
they are encoded and sent as a bitstream to video 
multiplexer 1207 (this encoded bitstream corre- 
sponds to the first video layer). MPEG-1 SIP encoder 

45 1 206 also outputs a decoded SIP odd signal to switch 
1208. When an odd field is being decoded, switch 

1208 allows the decoded SIP odd fields to be input to 
horizontal interpolator 1209. IHorizontal interpolator 

1209 up-samples the SIF odd fields to CCIR-601 re- 
50 solution, and outputs them to switch 1210. This up- 
sampled signal is passed by switch 1211 to field- 
structure CCIR-601 encoder 1211, where it is used as 
spatial prediction of the CCIR odd slices. Field- 
structure CCIR-601 encoder 1211 encodes the odd 

65 CCiR-601 fields of the digital video signal received 
via line 1202. with an adaptive selection between 
temporal and spatial picture predictions on a macro- 
block basis. Even CCIR-601 fields cannot be similarly 

10 
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encoded as even field SIF video is not available to 
provide a spatial prediction, so they are encoded by 
temporal prediction alone. The resulting encoded bll- 
streams, first for odd and then for even fields are out- 
put to video nnuKiplexer 1207. 

Video multiplexer 1207 multiplexes the encoded 
bilstreams and transmits them to video demultiplexer 
1212 vsrhere they are demultiplexed into individual blt- 
slreams. MPEG-1 SIF decoder 1213 decodes the bit- 
stream corresponding to the first video layer, thereby 
reconstituting an SIF odd field video signal which Is 
output on lines 1214 and 1215. When an odd field is 
being decoded, sv^itch 1216 allows the decoded SIF 
odd field on line 1215 to be input to horizontal inter- 
polator 1217. Horizontal interpolator 1217 up-sam- 
ples the odd fields received from decoder 1213 to 
CCIR-601 resolution. The up-sampled signal is 
passed by switch 1218 to field-structure CClR-601 
encoder 1219, where ft is used as spatial prediction of 
the CCIR odd slices. For odd fields, field-structuna 
CCIR-601 decoder 1219 decodes the bitstream cor- 
responding to the second video layer (employing an 
-optional spatlahpredlction-based upon-the interpolat- 
ed decoded SIF odd fields from layer one) to recon- 
stitute CCIR-601 odd f telds which are output on lino 
1220. For even fields. CCIR-601 field structure de- 
coder 1219 simply uses temporal predicUons from 
previously decoded CCIR-601 even f ields; no spatial 
prediction is used. 

A representation of the slice structure employed 
in the encoding performed within functional block 
1 200 of FIG. 12 is shown in FIG. 13. The odd and even 
fields are encoded one after the other. Each SIF odd 
slice is used to spatially predict the CCIR- 601 odd 
slice corresponding to the same spatial image. The 
slices of the SIF odd. CCIR-601 odd and CCIR-601 
even fields must be multiplexed as shown for the im- 
ages to be decoded correctly. 

Although FIG. 12 shows two separate encoders 
being employed to accomplish the encoding of the 
two video layers, the encoding for both layers may be 
performed sequentially wHhin a single adaptive en- 
coder. An example of such an adaptive encoder is 
shown in FIGS. 4A. 4B and 4C. The MPEG-1 encod- 
ing is performed in a nr^anner identical to the MPEG- 
1 (first layer) encoding of odd SIF fields for the previ- 
ously described three layer example the invention. 
The second layer encoding in this two-layer example 
of the invention is performed in a manner simflar to 
that described for the encoding of the CCIR-601 odd 
and even slices in the three layer example of the in- 
vention. However, since SIF even slices are not avail- 
able, the CCIR-601 even slices are encoded without 
spatial prediction. 

The decoding performed within decoder 1210 
may be accomplished by employing a decoder like 
that shown in FIG. 9. The process for decoding is the 
same as that described for decoding the low- resolu- 



tion SIF odd layer digital video signal (MPEG-1) In 
three layer example of the invention. 

The decoder shown in FIG. 11 may be employed 
to facilitate the decoding performed within decoder 
6 1214. The process for decoding is the similar to that 
described for decoding the high-resolution CCIR-601 
digital video signal in three layer example of th© In- 
vention. However, as SIF even slices are not avaflabb 
in this two layer example of the Invention, the CCIR- 
io 601 even slices are decoded without the benefit of 
spatial prediction. 

A third example of this invention is facilitated by 
the video signal encoding/decoding system illustrat- 
ed in FIG. 14. In this particular example tvwo layers of 
15 video are encoded: a f irst layer consisting of MPEG- 
1 compatibly encoded HHR frames, and a second lay- 
er consisting of CClR-60t frames encoded using 
adaptive choice of temporal prediction from previous 
coded pictures and spatial prediction (obtained S>y In- 
20 terpolating a decoded HHR layer) corresponding to 
current temporal reference. As shown, CCIR-601 dig- 
ital video signals are Input to functional blocCi 1400 vte 
--line-1402. The video input signals comprise a sucoos- 
ston of digital words, each representing soma Uifor- 
25 mation at a particular instant of time for a pel. A ccan- 
plete set of digital representations for an imag© at a 
particular instant of time is called a fram3. The digital 
video signal received on line 1402 to decln^ted to 
HHR video frames by decimator 1403. Such decin^ 
30 tors are well-known in the art Decimator 1403 out- 
puts the HHR video to MPEG-1 HHR encoder 1404, 
where it fe encoded and sent as a bitstream to video 
multiplexer 1405 (this encoded bitstream corre- 
sponds to the first video layer). MPEG-I HHR encod- 
30 er 1404 also outputs a decoded HHR video signal to 
horizontal interpolator 1406. Horizontal interpolator 
1406 up-samples the HHR f ields received from en- 
coder 1404 to CCIR-601 resolution. Franne-strucUiiro 
CCIR- 601 encoder 1407 encodes the CCIR-601 
40 frames of the digital video sfenal received via line 
1402, with an adaptive selection on a macroblocfic ba- 
sis between temporal and spatial picture predictions. 
The up-sampled signal output by horizontal interpola- 
tor 1406 is used for obtaining a spatial prediction, and 
45 an adaptive dual/Single field orframe prediction mode 
is used for temporal prediction. The resulting encoded 
bitstream (corresponding to the second video layer) 
is output by frame-structure CCIR-601 encoder 1407 
to video multiplexer 1405. 
50 Video multiplexer 1405 multiplexes the encoded 

bitstreams and transmits them to video demultiplexer 
1 408, where they are demultiplexed into individual bil- 
strea'ms. MPEG-1 HHR decoder 1409 decodes the 
bitstream corresponding to the first video lay®". 
55 thereby reconstituting an HHR video signal which fe 
output on lines 1410 and 1411. Horizontal interpolator 
1412 up-samples the HHR frames received from de- 
coder 1409 to CCIR-601 resolution. Frame-structure 
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CCIR-601 decoder 1413 decodes the bitstream cor- 
responding to the second video layer (employing a 
spatial prediction based upon the interpolated decod* 
ed layer one HHR frames received from interpolator 
1412) to reconstitute a CCIR- 601 video signal v^hich 6 
is output on line 1414. 

A representation of the slice structure employed 
in the encoding performed within functional block 
1400 of FIG. 14 Is shown In FIG. 16. The HHR and 
CCIR-601 frame slices are encoded alternately. io 
Since each CCIR-601 slice block makes use of the 
corresponding HHR slice blocks for optional spatial 
prediction, the HHR slice blocks are encoded first, be- 
fore the CCIR-601 slice blocks. Similarly, the HHR 
slice blocks are decoded first, before decoding the is 
corresponding CCIR-601 slice blocks. 

Although FIG. 14 shows two separate encoders 
being employed to accomplish the two layer video en- 
coding, the encoding for tx)th layers may be accom- 
plished sequentially within a single adaptive encoder. 20 
FIGS. 16A, 16B and 16C shows a simplified block di- 
agram of the internal architecture of such an adaptive 
encoder. As this encoder must perform two types of 
encoding, certain analyzers and sub-systenr« within it 
must be enabled and disabled as a function of the par- 25 
ticular type of encoding toeing performed at a given time. 
This enabling/dtsat>ling is performed as function of oon- 
trol signals generated by a mode controller. Mode con- 
troller 1700, shovm in FIG. 17, generates the signals to 
control the mode of operation for the encoder of FIGS. 30 
16A, 16B and 16C. Mode controller 1700 receives the 
same input signals as prevk>usly described mode controller 
500. The output of mode controller 1 700 provides b'hievel 
("(Tor "1") control signals including: hhr_slice f 1 " when the 
current slice is from an HHR frame); ccir_compat_slk5e 36 
("1* when current slice is of CCIR-601 resolution and 
uses a compatible prediction); and bypass control sig- 
nals disjnter, dis^spatial, dis_dual, dis_f ield^mc, and 
dis_field_code (v*/hich disable specific modes in the in- 
ter/intra, spatialAemporal prediction, dual/single field 40 
predictk)n, and field/frame prediction analyzers, and 
the field/frame coding detector). 

In performing the MPEG-1 HHR encoding (layer 
1), the HHR frame macroblocks output by decimator 
1403 (FIG. 14) are received by the adaptive picture 45 
encoder of FIGS. ISA, 16B and 16C via line 1601. As 
MPEG-1 encoding does not use spatial, field, or 
frame prediction, mode controller 1700 sets the 
dis_spatial, dis_dual, dis_field_mc, and dis_f ield_co- 
de signals to "1*. This effectively disables spatial/tem- so 
poral prediction analyzer 1602, dual/single field MCP 
analyzer 1603, field/frame motion compensated pre- 
diction ("field/frame MCP") analyzer 1 604, and places 
field/frame coding detector 1605 in a frame mode. As 
a result, the compatibility signal fs^t^comp*), the 55 
dual/single field motion compensation signal 
("dual/single_field_mc"), and the field/frame motion 
compensation signal ("field/frame_mc") are set to 



zero. In addition, the hhr_slice signal is set to '1", and 
the ccir_compat_slice signal is set to "O" by mode 
controller 1700. Respectively, these signals cause 
switch 1606 to close and switch 1607 to open. 

If the current picture is an l-picture. mode control- 
ler 1700 outputs a dis^inter bypass control signal of 
•1*. thereby disabling inter/intra analyzer 16088Othat 
it outputs a "0" inter/intra signal. This causes switch 

1609 to remain open so that differencer 1610 re- 
ceives a prediction signal of zero at its negative input. 
As a result, the input video signal on line 1601 is 
passed, unaltered, through differencer 1610 to block 
formatter 1611. Block formatter 1611 formats the re- 
ceived signal in frame block format (output on line 
1612) and field block format (output on line 1613). 
Sincef ield/frame_code signal is set to "O" switch 1614 
is placed in contact with line 1612. and the frame 
block formatted signal from line 1612 is passed to 
DCT 1615. If the current picture is not an l-picture. in- 
ter/intra analyzer 1608 is not disabled, but rather em- 
ployed to calculate the inter-codIng predtetion error 
and intra-coding error variances. Intra-coding error 
variance Is calculated by computing the square of the 
difference between the original video signal (received 
on line 1601) and a signal representing the average 
energy of the original video signal (the average ener- 
gy is computed t>y average computer 1616). Inter- 
coding prediction error is calculated by computing the 
square of the difference between the original video 
signal and an inter prediction signal received via line 
1617. An inter/intra signal is generated so as to select 
whichever of these computed variances represents 
the smaller energy. The resulting Inter/intra signal 
controls switch 1 609 and determines what differencer 

1610 will subtract from the video signal on line 1601 
before the signal is passed to block formatter 1611. 

DCT 1615 converts each 8x8 block into an 8x8 
matrix of transform coefficients and passes the coef- 
ficients along to forward quantizer 1618. Forward 
quantizer 1618 quantizes each coefficient based on 
whether the current macroblock is intra or inter cod- 
ed, and a quant-parmtr signal. This q uant- par mtr sig- 
nal is determined in light of the characteristics of the 
digital input signal communicated to quantizer adap- 
ter 1619 via line 1 620. t he fullness of buffer 1 621 , and 
in light of coding statistics compiled by coding statis- 
tics processor 1622. The quantized coefficients are 
forward scanned by forward scanner 1623 and 
passed to V/FWL encoder 1624. V/FWL encoder 
1624 encodes the quantized data and outputs it as a 
single bitstream to buffer 1621. 

In addition to transmitting encoded transform 
coefficients, V/FWL encoder 1624 also encodes and 
transmits a number of control signals and overhead 
data within the output bitstream, including: the quant- 
parmtr signal, the inter/intra signal, motion vectors, 
the picture_type signal, the s_t_comp signal, the 
dual/single_f ield.mc signal, the field/frame_mc sig- 
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naU the f ield/frame^code signal, and recon_ signals 
(which are employed in Ihe decoding of the bilsueam 
and will be discussed later). The V/FWL Encoder also 
provides coding slalislics information to coding stat- 
istics processor 1622. Buffer 1621 outputs the re- 
ceived bitstream at the appropriate rale for accep- 
tance by video multiplexer 1405 (FIG. 14). 

The HHR slice is locally decoded within the adap- 
tive decoder of FIGS. 16A. 16B and 16C via inverse 
scanner 1625. inverse quantizer 1626. IDCT 1627. 
and block unformatter 1628. The decoded signal is 
added by summing element 1629 to the prediction 
signal (evident at the negative input of differencer 
1610). The resulting signal is written to next picture 
store 1630 via write.next switch 1631, and to recon- 
structed HHR one-slice store ("recon slice store") 
1 632 via switch 1 606. The signal does not reach hor- ' 
izontal interpolator 1633 as switch 1607 is open (the 
cdr.compat.slice signal is set to "0". Before the en- 
coding of an I- or P- picture, the contents of next pic- 
ture store 1630 are transferred to previous picture 
store 1634 via write_prev switch 1635. 
^ _JIhe_5*«*«« wr ite next switch^ 631 and 
write^prev switch 1635 are controlled by read/write 
controller 1800 (FIG. 18). An independently running 
processor (not shown) provides read/write controller 
1800 with the following input signals: picture_struo- 
ture; slice_mux_orden and sl'ice.number. The output 
of read/write controller 1800 provides bi-level ("0" or 
••1°) control signals including: write^prev and 
wrUe^next signals (which enable writing to previous 
and next picture stores); and recon_ccir_write and re- 
con_ccir_read (which allow the storage and retrieval 
of reconstructed CCIR-601 frames within previous 
and next picture stores). 

FIG. 19 shows a simplified block diagram of a pic- 
ture store (1900). One such store is employed for stor- 
ing a previous picture, and another for storing a next 
picture wHhin the adaptive picture encoder of FIGS. 
16A, 16B and 16C. As shown, picture store 1900 con- 
tains a separate memory for CCIR-601 frames(1901) 
and HHR frames (1902). Picture store 1900 is conf ig- 
ured to allow the storage and retrieval of pictures In 
response to readAwrlte signals generated by 
read/write controller 1800. The recon^ccir^v^ite and 
recon.ccir.read signals are set to if the current 
video signal is CCIR. 

If the current picture being processed by the 
adaptive picture encoder of FIGS. 16A, 16B and 16C 
is a P- or B- picture, motion estimator 1636 generates 
frame, single field, and dual field motion vectors 
based on the picture.type signal, and a comparison 
of the HHR macroblocks input on line 1601 with the 
decoded reference HHR images in next and previous 
picture stores 1630 and 1634. The single and dual 
field motion vectors are not necessary in this mode, 
and only the frame motion vectors are sent to frame 
MO predictor 1638. Frame MC predictor 1638 calcu- 



lates predictions of the video on line 1601. The pre- 
diction output by frame MC predictor 1638 on line 
1639 is passed to switch 1609 via switches 1640 and 
1641 (dual/singlejield_mc and s_t_comp are set to 
5 "0"). This completes the MPEG-1 encoding of an HHR 
frame slice. 

As is shown in FIG. 15, the next slice to be encod- 
ed is a CCIR- 601 frame slice. For CCIR-601 encod- 
ing all analyzers are enabled (Le.. all "dls^" signals 
10 generated by mode controller 1700 are set to ^'O'*) and 
all modes are employed. The hhr^slice signal gener- 
ated by mode controller 1700 is set to "0". Differencer 
1610 subtracts either a prediction or ''0° (In the case 
of intra-coding) from the incoming CCIR-601 signal. 
16 and outputs the result to block formatter 1611. Blocfic 
formatter 1611 formats the received signal In frame 
block format (output on line 1612) and field block for- 
mat (output on line 1613). Field/frame coding detector 
1605 selects the field or frame fornf»atted block that 
20 contains the higher line correlation, and generates 
the appropriate f ield/frame_code to pass that block to 
DCT 1615, forward quantizer 1618, forward scannor 
1623, and V/FWL encoder 1624. The bitstream from 
V/FWL encoderT1^4~feW buffer 1821 fcx output 
26 to video multiplexer 1405 (FIG. 14). 

The output of the forward scanner Is locally do- 
coded by inverse scanner 1625, inverse quantlzeir 
1626, IDCT 1627, and block unformatter 1628. This 
decoded error signal is added to the predlctiort signal 
30 (evident at the negative input of differencer 1610) by 
summing element 1629. If an I- or P- picture is being 
encoded, switch 1631 is closed by reed/write control, 
ler 1800 and the output of summing element 1629 Is 
placed in next picture store 1 630. Prior to encoding en 
35 I- or P- picture, the contents of next picture store 1 630 
are transferred to the previous picture store 1634 vto 
write^prev switch 1635. 

A temporal prediction is arrived at as follows. Mo- 
lion estimator 1636 computes frame, single field, and 
40 dual field vectors based on the pidure.type control 
signal, and a best match of the CCIR- 601 macro- 
blocks input on line 1601 with the decoded referencaa 
images in next and previous picture stores 1630 and 
1634. The single and dual f ield motion vectors are sent 
46 to dual/single field MC predictor 1637. and the franta 
motion vectors are sent to frante MC predictor 1638. 
Both dual/single field MC predictor 1637 and frame MC 
predictor 1 638 calculate temporal predicUons of the vid- 
eo on line 1601. Dual/single field MCP analyzer 1603 
so selects either the dual or single f ield prediction, which- 
ever has lower energy in the error signal, and generates 
the appropriate dual/single_field_mc signal to cause 
that prediction to be passed to switch 1640 via switch 
1642. Field/frame MCP analyzer 1604 selects either 
55 the frame prediction from frame MC predictor 1638. or 
the field prediction provided by dual/single MCP ana- 
lyzer 1603, whichever has lower energy in the error 
signal, and generates the appropriate fieldAframe_n« 
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signal to cause that prediction to be passed to switch 
1641 via switch 1640. 

When the current CCIR-601 frame slice uses 
compatible prediction (ccir_compat_sIice « "I") 
switch 1607 is closed and the reconstructed HHR 
slice stored in recon slice store 1 632 is interpolated by 
horizontal interpolator 1633. The output of horizontal 
interpolator 1633, which is passed to switch 1641, 
provides a spatial prediction for the video input on line 
1601. 

SpatialAemporal prediction analyzer 1602 conrv 
pares the interpolated HHR signal (spatial prediction) 
with the signal provided by switch 1640 (temporal pre- 
diction) to determine which signal gives a better esti- 
mate of the Input video. Spatial/temporal prediction ana- 
lyzer 1602 outputs a compatibility signal (s_t_comp) 
which controls switch 1641 and allows the best choice 
of spatial or temporal prediction to be passed to in- 
ter/intra analyzer 1608. Inter/intra analyzer 1 608 then 
selects between intra-coding and inter-coding, and is 
to be used by comparing the variance of the signal on 
line 1601 to the variance of the inter prediction error 
signal computed using the prediction signal received 
via line 1617. An inter/intra signal is generated so as 
to cause switch 1609 to allow the appropriate signal 
corresponding to the lower of the two variances to t>e 
passed to differencer 1610. The inter/intra signal 
opens switch 1609 if intra-coding is selected, or 
closes it if inter-coding is selected. This completes the 
loop for coding CCIR-601 frame slice. 

As discussed at>ove, video multiplexer 1405 
(FIG. 14) receives the bitstreams related to the HHR 
and CCIR-601 slices. Video multiplexer 1406 multi- 
plexes these received signals and transmits them to 
video demultiplexer 1408 (FIG. 14), where they are 
demultiplexed and passed on to decoders 1409 and 
1413 (FIG. 14). 

The internal architecture of a picture decoder 
which facilitates the decoding of the MPEG-1 HHR 
digital video signal encoded by the encoder of FIGS. 
16A. 16B and 16C is shown in FIG. 20. This decoder 
includes buffer 2001, slice identifier 2002, switch 
2003, V/FWL encoder 2004, inverse scanner 2005, In- 
verse quantizer 2006, IDCT 2007, summing element 
2008. motion compensated predictor 2009, next pic- 
ture store 2010, and previous picture store 2011. The 
operation of this decoder is basically identical to that 
of the previously described MPEG-1 SIF odd decoder 
shown in FIG. 9. The primary difference being that 
components within this MPEG-1 HHR decoder are 
adapted to decode HHR frames as opposed to SIF 
odd fields. Switch 2003 responds to the hhr_slice sig- 
nal generated by slice identifier 2002, just as switch 
903 of the decoder of FIG. 9 responded to the sif.odd- 
_sllce signal. " 

FIG. 21 shows a simplified block diagram of the 
internal architecture of a picture decoder which facil- 
itates the decoding of a CCIR-601 digital video sig- 



nalencoded by the encoder of FIG. 16. This decod- 
er is able to decode both HHR and CCIR-601 
frame-structured layers. An incoming bitstream 
passes through buffer 2101 and slice identifier 
5 21 02. Slice identifier 21 02 scans t he bitstream for an 
hhr_slice_starl_code. Upon finding this code, the 
hhr^slice signal is set to "I* (indicative of an HHR 
slice). Slice identifier 2102 also scans the bitstream 
fora ccir_compat_slice_start_code, and upon finding 

10 it sets the ccir_compat_slice signal is set to (indi- 
cative of a CCIR-601 slice encoded using a compat- 
ible prediction). The bitstream is then passed to 
V/FWL decoder 2103. V/FWL decoder 2103 decodes 
the control signals and overhead data within the blt- 

16 stream, including: the picture_type signal, which is 
sent to dual/single field MC predictor 2104 and frame 
MC predictor 21 05; the motion vectors, which are sent 
to switch 2106; the inter/intra signal, which is sent to 
switch 2107 and inverse quantizer 2108; the 

20 quan_parmtr signal, which is sent to inverse quantizer 
2108; the dual/slngle_f leld_mc signal, which controls 
switches 2109 and 2110; the fleld/frame.mc signal, 
which controls switches 2106 and 2111; the sjt.oomp 
signal, which controls switch 2112; the ccir^oomp^slioe 

25 signal, which controls switch 2113; thef ield/rrame^CKie 
signal, which is sent to inverse scanner 2114 and 
block unfor matter 2115; and the fecon.cclr_write, 
and recon_ccir_read signals which are sent to next 
and previous pictures stores 2116 and 2117. Since 

30 this decoder can decode t>oth layers, the output may 
consist of HHR frames or CCiR-601 frames. 

The decoding of HHR slices is performed by this 
decoder in primarily the sanr>e manner as in the de- 
coder of FIG. 20, and will not be discussed in detal. 

36 However, the decoder of FIG. 21 performs one addi- 
tional step in HHR decoding. When the current slioe 
being decoded is an HHR slice, switch 211 8 is closed 
and a reconstructed HHR slice is stored in recon slice 
store 2119. This stored slice will be used as a basis 

40 for spatial prediction. 

The decoding of a CCIR-601 slice macroblock is 
performed when a ccir_compat_slice_start_code is 
detected by slice identifier 2102. Upon detection of 
this code the ccir_compat.slice signal is set to "1". In 

46 decoding a CCIR-601 slice macroblock, the V/FWL 
decoder outputs the coefficient related data of the er>- 
coded video to inverse scanner 2114. inverse quan- 
tizer 2108, IDCT 2120. and block unfor matter 2115. 
The output of block unformatter 2115 Is added, by 

60 summing element 2121, to a predictbn signal re- 
ceived from switch 2107. The resulting decoded 
CCIR-601 slice macroblock is written into next pk^tuie 
store 2116 via write_next switch 2122 if It is an I- era 
P- picture. Before decoding of I- or P- picture, the con- 
55 tents of next picture store 21 1 6 are transferred to pre- 
vious picture store 2117 via the closing of write_prev 
switch 2123. By controlling switch 2106, the 
f ield/frame_mc signal directs the motion vectors out- 

14 
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put by V/FWL decoder 2103 to either frame MC pre- 
dictor 2105 or dual/single f ield MC predictor 2104. If 
the motion vectors are routed to dual/single field MC 
predictor 2104, the dual/single_field_mc signal di- 
rects the motion vectors to the appropriate input of 
dual/single field MC predictor 2104. Dual/single field 
MC predictor 2104 then computes a prediction for the 
selected mode and passes that prediction to switch 
2111. If the motion vectors are routed to frame/field 
MC predictor 2105, then frame/field MC predictor 

2106 computes a prediction for the selected mode 
which is passed to switch 2111. Switch 2111 responds 
to the fieid/frame_mc signal to route the appropriate 
prediction to switch 2112. 

Switch 2113 is closed for all CCIR-601 slices as 
they employ spatial prediction (indicated by a 
ccir_compat_slice signal of "I"). When switch 2113 Is 
closed, the HHR slice in recon slice store 2119 is in- 
terpolated by horizontal interpolator 2124 and passed 
to switch 2112. Switch 2112 allows either the interpo- 
lated reconstructed HHR slice, or the estimate sent 
from switch 2111 to be used as a spatial prediction for 
the current CCIR-601 signal (depending upon the 
sZLcomfTsignar 

If an inter-coded picture is being decoded, switch 

2107 passes the prediction to summing element 
2121, where it is added to the signal output by block 
unformatter2115. However, if the current picture be- 
ing decoded is intra coded (inter/intra set to "0", and 
switch 2107 opened), a signal of zero is added to the 
reconstituted macroblock passing through summing 
element 2121. 

Yet another example of this invention is facilitated 
by the video signal encoding/decoding system illu- 
strated in FIG. 22 which encodes and decodes of a 
digital video signal having three layers of spatial re- 
solution. Signal encoding is performed within func- 
tional block 2200, and signal decoding is performed 
within functional block 2201. Layer 1 consists of 
MPEG-1 encoded SIF odd fields and produces an 
MPEG-1 compatible bitstream. Layer 2 consists of 
SIF even fields encoded using adaptive dual/single 
field nnotion compensated predictfons. Layer 3 con- 
sists of CCIR- 601 frame-structured pictures which 
are encoded using an adaptive choice of temporal 
prediction from previously decoded pictures of the 
same resolution, and a spatial prediction correspond- 
ing to a current temporal reference obtained by inter- 
polating decoded SIF resolution layers 1 and 2. Since 
layer 3 is frame-structured, it may use adaptive 
frame/dual/single field prediction modes for temporal 
prediction. 

As shown in FIG. 1, CCIR-601 digital video sig- 
nals are input to functional block 2200 via line 2202. 
The digital video signal received on line 2202 is ded- 
maled to SIF video by SIF decimator 2203 for the en- 
coding of layers 1 and 2. SIF decimator 2203 outputs 
decimated odd numbered horizontal rows via line 



2204, and even numbered horizontal rows via Una 

2205. The odd horizontal rows comprise an odd video 
field, the even horizontal rows comprise an even vid- 
eo field. The SIF odd fields are directed to MPEG-1 

6 SIF encoder 2206. where they are encoded and sent 
as a bitstream to video multiplexer 2207 (this enood- 
ed bitstream corresponds to tho first video layer). 
MPEG-1 SIF 2206 encoder also outputa a decoded 
SIF odd signal to even field SIF encoder 2208 and 
10 horizontal interpolator 2209 via lines 2210 and 2211, 
respectively. Even field SIF encoder 2208 encodes 
the SIF even fields into a bitstream (corresponding Go 
I he second video layer) which Is output to video nmil- 
liplexer 2207. Even field SIF encoder 2208 also out- 
16 puts a decoded SIF even signal to horizontal interpo- 
lator 2209 via line 2212. Horizontal Interpolator 220& 
up-samples the odd and even fields received from en- 
coders 2206 and 2208 to CCIIR-601 resdutton. Thfe 
up-sampled signal is output via lino 2213 to franno 
20 structure CCIR-601 encoder 2214. whoro K us(sd 
for obtaining a spatial prediction of tha fthbnd vWoo Isy- 
er. Frame-structure CCIR-^01 encodsy 2214 ©fi- 
codes the CCIR-601 frames of the dlgltsJ vkJeo slgnol 
receiveOVia irrie^027witlTl^ ©©- 
25 lection between temporal and spatial predlcftkms. Tha 
resulting encoded bitsUeam (ccrrespondSng to thd 
third video layer) is output by frame-structiwo CCIIR- 
601 encoder 2214 to video multiple»Qir 22@7. 

Video multiplexer 2207 multiple»ea th© three ©n- 
30 coded bitstreams into a single bitstream. Thla single 
bitstream is transmitted to video demultiplexer 2215 
where it is demultiplexed into thre® Individual bll- 
streams, one for each of the thre® video layers. 
MPE6-1 SIF decoder 2216 decodes th© bitstream 
36 corresponding to the first video layer, thereby recon- 
structing an SIF odd field video signal which Is output 
on lines 2217, 221 8, and 2219. Even field SIF decod- 
er 2220 decodes the bitstream corresponding to th® 
second video layer (employing an optional blocfic-wis® 
40 prediction based upon the SIF odd fields decoded by 
MPEG-1 SIF decoder 2216). This yields an SIF even 
field video signal which is output on lines 2221 and 
2222. This SIF even field video may be re-interlacsd 
with the SIF odd field video output by MPEG-1 SIIF 
46 decoder 2216 to obtain HHR resolution. Horizontal In- 
terpolator 2223 up-samples the odd and even fields 
received from decoders 2216 and 2220 to CCIR-601 
resolution. Frame-structured CCIR-601 decoder 
2224 decodes the bitstream corresponding to tte 
50 third video layer (employing an optional spatial pre- 
diction based upon the interpolated decoded SIF odd 
and even fields from layers one and two) to reconst^ 
tute a CCIR-601 video signal which is output on Una 
2225. 

55 A representation of the slice structure employed 

in the encoding performed within functional blodk 
2200 of FIG. 22 is shown in FIG, 23, The frame- 
structure coding of layer 3 requires odd arul even 
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fields of a frame to be presented to the encoder at the 
same time (as opposed to one field after another). 
Therefore, the SIF odd and even f ield slices have to 
be coded and multiplexed alternately. The CCtR-601 
slices are frame slices; i.e., each slice contains lines 
from both odd and even fields. The SIF odd fields are 
coded referencing SIF odd fields only (MPEG-1), but 
the SIF even fields are coded referencing both SIF 
odd and even fields for temporal prediction. To en- 
code a SIF even slice, three contiguous SIF odd slic- 
es must be available to cover the motion range re- 
quired for temporal prediction of the SIF even slice. 
This necessitates encoding two SIF odd slices at the 
start of each frame. Encoding of the first SIF even 
slice within a frame requires only two reconstructed 
odd slices. The two SIF odd slices and one SIF even 
slice provide enough reconstructed data for the spa- 
tial prediction of two corresponding CCIR-601 frame 
slices. Following the encoding of the two CCIR-601 
frame slices, one SIF odd slice is encoded, which to- 
gether with the two previous SIF odd slices provides 
enough temporal reference for the second SIF even 
slice. The second SIF odd and even slices can be 
used for spatial prediction for the third and fourth 
CCIR- 601 frame slices. At this point in the frame, the 
slice pattern of SIF odd/SIF even/CCIR-601 
frame/CCIR-601 frame is repeated until the end of the 
frame is reached. However, there is a slight irregular- 
ity in this pattern at the very end of a frame. Within 
the final set of slices in a frame the SIF odd slice is 
omitted. This compensates for the two contiguous 
SIF odd slices which were are encoded at the begin- 
ning of each frame. 

Although FIG. 22 shows three separate encoders 
being employed to accomplish the encoding of the 
three video layers* the encoding for all three layers 
may be accomplished sequentially within a single 
adaptive encoder. FIGS. 24A, 24B and 24C shows a 
simplified block diagram of the internal architecture of 
one example of such an adaptive picture encoder. As 
this encoder must perform three distinctly different 
types of encoding, certain analyzers and subsystems 
within it must be enabled and disabled as a function 
of the particular type of encoding being performed at 
a given time. This enabling/disabling is performed as 
function of control signals generated by a mode con- 
troller. Mode controller 2500, shown in FIG. 25, gen- 
erates the signals to control the mode of operation for 
the encoder of FIGS. 24A, 24B and 240. Mode control- 
ler 2500 receives the sarr^ input signals as previously 
described mode controllers 500 and 1700. The output of 
mode controller 2500 provides bUevel ("0" or "1") control 
signals including: sif_odd_slice ("1" when the current 
slice is from an SIF odd field); sif_even_slice ("I* 
when the current slice is from an SIF even field); 
ccir_compat_slice ("1" when current slice is of CCIR- 
601 resolution and uses a compatible prediction); and 
bypass control signals dis_inter, dis_spatial, dis.dual, 



dis_f ield_mc, and dis_f ield_code (which disable spe- 
cific modes in the inter/intra. spatialAemporal predic- 
tion, dual/single field prediction, and field/frame pre- 
diction analyzers, and a field/frame coding detector). 

6 In performing the MPEG-1 SIF odd encoding 

(layer 1). the SIF odd macroblocks output by decima- 
tor 2203 (FIG. 22) are received by the adaptive pic- 
lure encoder of FIGS. 24A, 24B and 24C via line 2401. 
As MPEG-1 encoding does not use spatial prediction, 

10 dual-field prediction, frame prediction, or field coding, 
mode controller 2500 sets the dis.spatial, dis^dual, 
and dis_f ield_code signals to "1*. This effectively dis- 
ables spatial/temporal prediction analyzer 2402, 
dual/single field MCP analyzer 2403, field/frame MOP 

16 analyzer 2404, and places field/frame coding detector 
2405 in a frame mode. As a result, the compatbHity signal 
("s_t_comp"), the dual/single field motion compensation 
signal ("duaUsingle_f ield^nrx:"), and the field/frame mo- 
tion compensation signal fTield/finame^mcT) are set to 

20 zero. In additiori, the sff_odd_slice signal is set to "1*. 
and the cclr_compat_sIlce signal is set to "O* by mode 
conUoller 2500. Respectively, these signals cause 
"switch 2406 to close arid switch 2407 to open. 

If the current picture is an l-picture, mode control- 

25 ler 2500 outputs a dis_inter bypass control signal of 
"1", thereby disabling inter/intra analyzer 2408 so that 
it outputs a -0" inter/intra signal. This causes swKch 

2409 to remain open so that differencer 2410 re- 
ceives a prediction signal of zero at its negative input 

90 As a result, the input video signal on line 2401 is 
passed, unaltered, through differencer 2410 to block 
formatter 2411. Block formatter 2411 formats the re- 
ceived signal in frame block format (output on line 
2412) and field block fornriat (output on line 2413). 

3B Since f ield/frame_code signal is set to switch 2414 
is placed in contact with line 2412, and the frame 
block formatted signal is passed to DOT 2415. If the 
current picture is not an l-picture, inter/intra analyzer 
1 608 is not disabled, but rather employed to calculate 

40 the inter-coding prediction error and intra-coding er- 
ror variances. Intra-coding error variance is calculat- 
ed by comparing the square of the difference t>e- 
t ween t he original video signal (received on line 2401) 
and a signal representing the average energy of the 

46 original video signal (the average energy is computed 
by average computer 2416). Inter-coding prediction 
error variance is calculated by computing the square 
of the difference between the original video signal 
and an inter prediction signal received via line 2417. 

50 An inter/intra signal is generated so as to select 
whichever of these computed variances represents 
the smaller energy. The resulting inter/intra signal 
controls switch 2409 and determines what differencer 

2410 will subtract from the video signal on line 2401 
55 before the signal is passed to block formatter 2411 . 

DCT 2415 converts each 8x8 block into an 8x8 
matrbc of transform coefficients and passes the coe^ 
ficients along to forward quantizer 2418. Forward 
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quantizer 2418 quantizes each coefficient based on 
whether the current macroblock is intra or inter cod- 
ed, and a quant-par mtr signal. This quant-par mtr sig- 
nal is determined in light of the characteristics of the 
digital input signal communicated to quantizer adap- 
ter 2419 via line 2420, the fullness of buffer 2421. and 
in light of coding statistics compiled by coding statis- 
tics processor 2422. The quantized coefficients are 
forward scanned by forward scanner 2423 and 
passed to V/FWL encoder 2424 . V/FWL encoder 2424 
encodes the quantized data and outputs it as a single 
bitstream to buffer 2421. 

In addition to transmitting encoded transfornn 
coefficients, V/FWL encoder 2424 also encodes and 
transmits a number of control signals and overhead 
data within the output bitstream, including: the quant- 
parmtr signal, the inter/lntra signal, motion veclofs. 
the picture_type signal, the s_t_comp signal, the 
dual/single_field_mc signal, the f ield/frame_mc sig- 
nal, the f ieTd/frame_code signal, and recon_ signals 
(which are employed In the decoding of the bitstream 
and will be discussed later). The V/FWL Encoder also 
provides coding statistics information. to coding stat- 
istics processor 2422. Buffer 2421 outputs the re- 
ceived bitstream at the appropriate rate for accep- 
tance by video multiplexer 2207 (FIG. 22). 

The SIF odd slice Is locally decoded within the 
adaptive decoder of FIGS. 24A, 24B and 24C via In- 
verse scanner 2425, inverse quantizer 2426, IDCT 
2427, and block unformatter 242a. The decoded sig- 
nal is added by summing element 2429 to the predic- 
tion signal (evident at the negative input of differencer 
2410). The resulting signal is written to next picture 
store 2430 via write_next switch 2431 (if it is an I- or 
a P- picture), and to reconstructed SIF odd two-slice 
store (•'odd slice store") 2432 via switch 2406. Odd 
slice store 2432 has to be capable of storing two SIF 
odd slices because at the beginning of every new 
frame, two odd slices must be encoded before they 
are used for spatial prediction of the CCIR-601 layer. 
Before the encoding of an 1- or P- picture, the con- 
tents of next picture store 2430 are transferred to pre- 
vious picture store 2434 via write_prev switch 2435. 

The states of write.next switch 2431 and 
write_prev switch 2435 are controlled by read/write 
conUoller 2600 (FIG. 26). An independently running 
processor (not shown) provides read/write controller 
2600 with the following input signals: piclure.struo- 
ture; slice_mux_orden and slice_number. The output 
of read/write controller 2600 provides bi-ievel ("C or 
"1*^ control signals including: write_prev and 
write_next signals (which enable writing to previous 
and next picture stores); recon_odd_wrile and re- 
con_odd_read (which allow the storage and retrieval of 
reconstructed SIF odd fields within previous and next 
picture stores); and recon_ccirjwrite and recon_cdr_re- 
ad (which allow the storage and retrieval of recon- 
structed CCIR-601 frames within previous and next 



picture stores). 

FIG. 27 shows a simplified block diagram of a pic- 
ture store (2700). One such store is employed for stor- 
ing a previous picture, and another for storing a next 
6 picture within the adaptive picture encoder of FIGS. 
24 A, 24 B and 24C. As shown, picture store 2700 con- 
tains separate memories for CCIR-601 frames 
(2701), SIF odd fields (2702), and SIF even fields 
(2703). Picture store 2700 is configured to allow the 
10 storage and retrieval of pictures In response to 
read/write signals generated by read/write controller 
2600. The recon_ccir_write and recon_cclr_read sig- 
nals are set to if the current video signal is CCIR, 
otherwise they are set to "0^ Likewise, the re- 
ts con_odd_write a nd recon_odd_read signals are set to 
-1- if the current video signal is SIF odd, otherwise 
they are set to t)**. 

If the current picture being processed by the 
adaptive picture encoder of FIGS. 24A, 248 and 24C 
20 is a P- or B- picture, motbn estimator 2436 generates 
frame, single field, and dual field motton vectors 
based on the plcture_type signal, and a comparison 
of the SIF odd n^acroblocks Input on line 2401 with the 
decoded reference images in next and prevtous pk:tine 
2S stores 2430 and 2434. The single and dual field motion 
vectors are sent to dual/single field MC predtelor 2437, 
and the frame motkm vectors are sent to frame MC py©- 
didor 2438. Both dual/single field MC predictor 2437 
and frame MC predictor 2438 calculate prediclions of 
30 t he video on line 2401 . However, only the prediction out- 
put by dual/single field MC predictor 2437 on line 2439 
is passed to switch 2409 via switches 2442, 2440 and 
2441 (dis_dual and dts_fiel decode being set to 
and 8_t_comp being set to °0^. Thte completes the 
36 MPEG-1 encoding of an SIF odd slice. Although, ©t 
the beginning of a frame, this encoding would l>e re- 
peated, as the encoding of two successive SIF odd 
slices would be required. 

The SIF even encoding performed by the enc«S- 
40 er of FIGS. 24A, 24B and 24C Is very similar to that 
described above for the SIF odd slices. The primary 
difference between the two encoding processes is 
that the sif_even_slice signal is set to ''I'*, and the 
sif_odd_slice signal is set to "0° by mode controller 
45 2500. As a result, the signal output by summing ele- 
ment 2429 is written to next picture store 2430 v'a 
write_next switch 2431 , and to reconstituted SIF even 
one-slice store ("even slice store'O 2443 via switch 
2406. 

60 Upon completion of the encoding of the SIF odd 

and SIF even slices, a CCIR-601 frame slice is encod- 
ed. For CCIR-601 encoding all analyzers are enabled 
(i.e., all "dis.** signals generated by mode controller 
2500 are set to "0^ and all modes are employed. The 

55 sif_odd_slice and sif_even_slice signals generated 
by mode controller 2500 are set to "O*". Differencer 
2410 subtracts either a prediction or "0° (in the case 
of intra-coding) from the incoming CCIR-601 signal 
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on line 2401, and outputs the result to block formatter 
2411. Block formatter 2411 formats the received sig- 
nal in frame block format (output on line 2412) and 
field block format (output on line 2413). Field/frame 
coding detector 2405 selects the field or frame for- 5 
matted block that contains the higher line correlation, 
and generates the appropriate field/frame_code to 
pass that block to DCT 2415, forward quantizer 2418, 
forward scanner 2423, and V/FWL encoder 2424. The 
bitstream from V/FWL encoder 2424 is fed to buffer to 
2421 for output to video multiplexer 2207 (FIG. 22). 

The output of the forward scanner is locally de- 
coded by inverse scanner 2425. inverse quantizer 
2426, IDCT 2427. and block unformatter 2428. This 
decoded error signal is added to the prediction signal i6 
(evident at the negative input of differencer 2410) by 
summing element 2429. If an I- or P- picture is being 
encoded, switch 2431 is closed by read/write control- 
ler 2600 and the output of summing element 2429 is 
placed in next picture store 2430. Prior to encoding an 20 
I- or P- picture, the contents of next picture store 2430 
are transferred to the previous picture store 2434 via 
"Wfliw^^pr^v^W - 

A temporal prediction is arrived at as follows. Mo- 
tion estimator 2436 computes frame, single field, and 25 
dual field vectors based on the picture-type signal, 
and a comparison of the CCIR-601 macroblocks input 
on line 2401 with the decoded reference images In 
next and previous picture stores 2430 and 2434. The 
single and dual field motksn vectors are sent to 30 
dual/single field MC predictor 2437, and the frame 
motion vectors are sent to frame MC predictor 2438. 
Both dual/single field MC predictor 2437 and frame 
MC predictor 2438 calculate predictions of the video 
on line 2401. Dual/single field MOP analyzer 2403 se- 3S 
lects eKher the dual or single field prediction (which- 
ever contains a lower energy in the error signal), and 
generates the appropriate dual/single_field_mc sig- 
nal to cause that prediction to be passed to switch 
2440 via switch 2442. Field/frame MCP analyzer 2404 40 
selects either the frame prediction from frame MC 
predictor 2438, or the best of the field predictions pro- 
vided by dual/single MCP analyzer 2403 (whichever 
contains a lower energy in the error signal), and gen- 
erates the appropriate f ield/frame_mc signal to cause 4S 
that prediction to be passed to switch 2441 via switch 
2440. As the dis_field_code signal is set to "O* by 
mode controller 2500, the frame_f ield_mc signal out- 
put by field/frame MCP analyzer 2404 will determine 
the output of OR-gate 2444. and therefore the state 50 
of switch 2440. 

When the current CCIR-601 frame slice uses 
compatible prediction (ccir_compat_slice = "1") 
switch 2407 is closed and a reconstructed HHR slice 
from HHR interleaver 2445 is passed to horizontal Irv 55 
terpolator 2446. HHR interleaver2445 constructs this 
HHR slice from the contents of odd slice store 2432 
and even slice store 2443. The output of horizontal in- 
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terpolator 2446, which is passed to switch 2441 and 
spatial/temporal prediction analyzer 2402. provides a 
spatial prediction for the video input on line 2401. 

Spatial/temporal prediction analyzer 2402 com- 
pares the interpolated HHR signal (spatial prediction) 
with the signal provided by sv^tch 2440 (temporal pre- 
diction) to determine which signal gives a t>etter esti- 
mate of the input video. Spatial/temporal prediction ana- 
lyzer 2402 outputs a compatibility signal (8_t_comp) 
which controls switch 2441 and allows the best esti- 
mate to be passed to inter/intra analyzer 2408. In- 
ter/intra analyzer 2408 then selects between Intra- 
coding and inter-coding, and by computing the vari- 
ance of the signal on line 2401 and the variance of the 
inter prediction signal received via line 2417. An In- 
ter/intra signal is generated so as to cause switch 
2409 to allow whichever of the best prediction to be 
passed to differencer 2410. The inter/intra signal 
opens switch 2409 if intra-coding Is selected, or 
closes it if inter-coding is selected. This completes the 
loop for coding CCIR-601 frame slice. However, since 
odd slice sjore 2432 and even slice store 2443 contain 
"eh^Ogh data foT spatia% predicting two CCIR-601 
frame slices, two CCIR-601 slices are consecutively 
encoded before encoding of the next SIF odd slice. 

As discussed above, video multiplexer 2207 
(FIG. 22) receives the bitstreams related to the SIF 
odd, SIF even and CCIR-601 slices. Video multiplexer 
2207 multiplexes these received signals and trarw- 
mits them to video demultiplexer 2215 (FIG. 22) 
where they are demultiplexed and passed on to de- 
coders 2216. 2220, and 2224 (FIG. 22). 

The decoding of the SIF odd slices encoded by 
the encoder of FIGS. 24A. 24B and 24C is performed 
by decoder 2216 (FIG. 22). As both the configuration 
and operation of this decoder are identical to that of 
the decoder illustrated in FIG. 9, the specifics of this 
decoding wll not be discussed. Liicewise. since the 
configuration and operation of SIF even decoder 
2220 are identical to that of the decoder Illustrated in 
FIG. 10, a detailed discussion of the operation of de- 
coder 2220 is unnecessary. 

FIG. 28 shows a simplified block diagram of the 
internal architecture of a picture decoder which fad- 
Itates the decoding of a CCIR-601. SIF odd. and SIF 
even digital video signals encoded by the encoder of 
FIG. 24. An incoming bitstream passes through buffer 
2801 and slice identifier 2802. Slice identifier 2802 
scans the bitstream for an sif_odd_slice_start code 
and an sif_even_slice_start_code. Upon finding 
these codes, the sif_odd_slice signal is set to "I* (in- 
dicative of an SIF odd slice) and the sif_even_slloe 
signal is set to "1" (Indicative of an SIF even slice). 
Slice identifier 2802 also scans the bitstream for a 
ccir_compat_slice_start_code. Upon finding this 
code the ccir_compat_slice signal is set to M* (indi- 
cative of a CCIR-601 slice encoded using a compat- 
ible prediction). The bitstream is then passed to 
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V/R/VL decoder 2803. V/FWL decoder 2803 decodes 
the conuol signals and overhead data within the bit- 
stream, induding: the picture_lype signal, which is 
sent to dual/single field MC predictor 2804 and frame 
MC predictor 2805; the motion vectors, which are sent 
to switch 2806; the inter/inlra signal, which is sent to 
switch 2807 and inverse quantizer 2808; the 
quan_parmtr signal, which is sent to inverse quantiz- 
er 2808; the dual/single_f ield_mc signal, which con- 
trols switches 2809 and 2810; the field/frame_mc 
signal, which controls switches 2806 and 2811; the 
s_t_comp signal, which controls switch 2812; the 
cdr.compat_slice signal, which controls switch 2813; 
the fieldAframe_code signal, which is sent to inverse 
scanner 2814 and block unformatter 2815; and the re- 
con_odd_write, recon_odd_read, recon_ccir_write, 
and recon_ccir_read signals which are sent to next 
and previous pictures stores 2816 and 2817. 

The decoding of SIF odd and even slices is per- 
formed by this decoder In primanly the same manner 
as in the decoders of FIGS. 9 and 10, respectively, 
and will noi be discussed In detail. However, the de- 
coder of FIG. -28 performs an.addltional step in SIF 
odd and even decoding. When the current slice being 
decoded is an SIF odd slice. swilch2818 is closed and 
a reconstruction of the SIF odd slice is stored in re- 
constituted SIF odd two-slice store ("odd slice store") 
2819. When the current slice being decoded is an SIF 
even slice, switch 2820 is dosed and a reconstruction 
of the SIF even slice is stored in reconstituted SIF 
even slice store ("even slice store") 2821. These stor- 
ed slices wfll be used as a basis for spatial prediction. 

The decoding of a CCIR-601 slice macroblock is 
performed when a ccir_compat_slice_start_code is 
detected by slice identifier 2802 (ccir_compat_slice is 
set to "1"). In decoding a CCIR-601 slice macroblock, 
the bitstream is passed through inverse scanner 
2814. inverse quantizer 2808. IDCT 2822, and block 
unformatter 2815. The output of block unformatter 
2815 is added, by summing element 2823, to a pre- 
diction signal received from switch 2807. The result- 
ing decoded CCIR-601 slice n^croblock is written 
into next picture store 2816 via write_next switch 2824 
if it is an I- or a P- picture. Before decoding of I- or P- 
picture, the contents of next picture store 2816 are 
transferred to previous picture store 2817 via the 
closing of write_prev switch 2825. By controlling 
switch 2806. the field/frame_mc signal directs 
the motion vectors output by V/FWL decoder 2803 
to either frame MC predictor 2805 or dual/single 
field MC predictor 2804. If the motion vectors are 
routed to dual/single field MC predictor 2804, the 
dual/single_field_mc signal directs the motion vec- 
tors to the appropriate input of dual/single field MC 
predictor 2804. Dual/single field MC predictor 2804 
then computes a prediction for the selected mode and 
passes that prediction to switch 2811. If the motion 
vectors are routed to frame/field MC predictor 2805, 



then frame/field MC predictor 2805 computes a pre- 
diction for the selected mode whteh Is passed to 
switch 2811. Switch 2811 responds to the 
f ield/frame_mc signal to route the appropriate predlo- 
6 tion to switch 2812. 

Switch 2813 is closed for all CCIR-601 slices 
which employ spatial prediction (Indicated by a 
ccir_compat_slice signal of "I"). When switch 2813 is 
closed, a reconstructed HHR slice from HHR inter- 
to leaver 2826 is passed to horizontal Interpolator 2827. 
HHR interleaver 2826 constructs this HHR slice from 
the contents of odd slice store 2819 and even slice 
store 2821 . The output of horizontal interpolator 2827, 
which is passed to switch 281 2, provides a spatial pre- 
16 d iction for t he video in put to buffer 2801. Switch 2812 
allows either the interpolated reconstructed HHR 
slice, or the estimate sent from switch 2811 to be used 
as a spatial prediction for the current CCIR-601 signal 
(depending upon the s_t_comp signal decoded by 
20 V/FWL decoder 2803). If an inter-coded picture Is be- 
ing decoded, switch 2807 passes the prediction data 
to summing element 2823 where It Is added to the sig- 
nal output by blqck^unfqrmatter M However. If the 
current picture being decoded Is Intra coded (Inter/ln- 
26 Ira set to "0", and switch 2807 opened), a signal of 
zero Is added to the reconstituted macroWock pass* 
ing through summing element 2823. Sln€» this de- 
coder can decode all three tayera, the output may 
consist of SIF odd. SIF even, or CCIR-60i rnacro- 
30 blocks. 

The above-described invention provides a prac- 
tical method for encoding and decoding video signals 
in a manner which allows the transmission, reception, 
storage, or retrieval of a video signal providing multl- 
36 pie layers of resolution. It will be understood that the 
particular methods described are only illustrative of 
the principles of the present invention, and that vari- 
ous modifications could be made by those skilled in 
the art without departing from the scope and spirit of 
40 the present invention, which is limited only by the 
claims that follow. Such modifications would include 
encoding and decoding schemes invohfing more than 
three resolution layers, or schemes that apply the 
principles of the invention to code progressive 0.e,. 
46 non-interlaced) video (such as that being proposed 
as an HDTV standard). 



Claims 

50 

1. Apparatus for encoding digital video signals, 
comprising: 

means for receiving a digital video input 
signal including a succession of digital represen- 
55 tations related to picture elements of a first video 

image; 

means for coding a reduced resolution dig- 
ital signal related to the picture elements of said 
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first video image, employing in said coding, if said 
first video image is not the initial image for which 
an input signal was received, a prediction of said 
first video image based upon a previously coded 
video image from a previously received input sig- 5 
nal; 

means for producing a temporal prediction 
of said f irst video image from said reduced reso- 
lution digital signal; 

means for producing a spatial prediction of io 
said first video image based upon said temporal 
prediction produced from said reduced resolution 
digital signal; and 

means for coding a second digital signal 
related to the picture elements of said first video 15 
image, adapted to determine if an estimate based 
upon said temporal prediction of said first video 
Image, or an estimate based upon said spatial 
prediction of said first video Image will be em> 
ployed in the encoding of said second digital sig- 20 
nal. 

Method for encoding digiUil vldeb'^slgnals, com- 
prising the steps of: 

receiving a digital video input signal indud- 26 
ing a succession of digital representations related 
to picture elements of a first video image; 

coding a reduced resolution digital signal 
related to the picture elements of said first video 
image, employing in said coding, if said first video so 
Image is not the initial image for which an input 
signal was received, a prediction of said first vid- 
eo image based upon a previously coded video 
image from a previously received input signal; 

producing a temporal prediction of said 36 
first video image from said reduced resolution 
digital signal; 

producing a spatial prediction of said first 
video image based upon said temporal prediction 
produced from said reduced resolution digital sig- 40 
nal; and 

coding a second digital signal related to 
the picture elements of said first video image, 
adapted to determine If an estimate based upon 
said temporal prediction of said first video image, 46 
or an estimate based upon said spatial prediction 
of said first video image will be employed in the 
encoding of said second digital signal. 

Apparatus for encoding d'^ital video signals, 60 
comprising: 

means for receiving a digital video input 
signal including a succession of digital represen- 
tations related to picture elements of at least two 
frames of a video image, each of said frames 55 
comprising a plurality of interlaced fields; 

means for coding a reduced resolution dig- 
ital signal related to the picture elements of a first 



field of a received frame, employing in said cod- 
ing. If said received frame is not the initial UBwe 
received, a prediction of said first field based 
upon a previously coded field from a previously 
received frarT>e; 

means for producing a temporal prediction 
of said first field from said reduced resolution dig- 
ital signal; 

means for producing a spatial prediction of 
said first video field based upon said temporal 
prediction produced from said reduced resolution 
digital signal; and 

means for coding a second digital signal 
related to the picture elements of said first video 
field, adapted to determine If an estimate based 
upon said temporal prediction of said first video 
field, or an estimate based upon said spatial pre- 
diction of said first video field will be employed in 
the encoding of said second digital signal. 

4. Method for encoding digital video signals, com- 
prising the steps of: 

^ " "^"^^fecelvlrig a^igltalvlde^ Includ- 
ing a succession of digital representations related 
to picture elements of at least two frames of a vid- 
eo image, each of said frames comprising a plur- 
alKy of interlaced fields; 

coding a reduced resolution digital signiri 
related to the picture elements of a first field of a 
received frame, employing in said coding, if said 
received frame is not the first initial received, a 
prediction of said first field based upon a previ- 
ously coded field from a previously received 
frame; 

producing a temporal prediction of said 
first field from said reduced resolution digital sig- 
nal; 

producing a spatial prediction of said first 
video field based upon said temporal prediction 
produced from said reduced resolution digital sig- 
nal; and 

coding a second digital signal related to 
the picture elements of said first video field, 
adapted to determine if an estimate based upon 
said temporal prediction of said first video field, 
or an estimate based upon said spatial prediction 
of said first video field will be employed in the en- 
coding of said second digital signal. 

5. Apparatus for encoding digital video signals, 
comprising: 

means for receiving a digital video input 
signal including a succession of digital represen- 
tations related to picture elements of at least two 
frames of a video image; 

means for processing said received digital 
video input signal so as to obtain a signal repre- 
senting a reduced resolution image of the re- 
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ceived frame; 

means for coding a reduced resolution dig- 
ital signal related to the picture elements of said 
received frame, employing in said coding, if said 
received frame is not the initial frame received, a 5. 
prediction of said received frame based upon a 
previously coded reduced resolution image of a 
previously received frame; 

means for producing a temporal prediction 
of said received frame from said reduced resolu- to 
tion digital signal; 

means for producing a spatial prediction of 
said received frame based upon said temporal 
prediction produced from said reduced resolution 
digital signal; and 

means for coding a second digital signal 
related to the picture elements of said received 
frame, adaptively employing an estimate based 
upon said temporal and said spatial predictions of 
said recehfed frame. 

6. Method for encoding digital video signals, com- 

prising -t he-steps of: 

receiving a digital video input signal includ- 
ing a succession of digital representations related 25 
to picture elements of at least two frames of a vid- 
eo image; 

processing said received digital video irv 
put signal so as to obtain a signal representing a 
reduced resolution image of the received franne; so 

coding a reduced resolution digital signal 
related to the picture elements of said received 
frame, employing in said coding, if said received 
frame is not the initial frame received, a prediction 
of said received frame based upon a previously 36 
coded reduced resolution image of a previously 
received frante; 

producing a temporal prediction of said re- 
ceived frame from said reduced resolution digital 

signal; ^ 

producing a spatial prediction of said re- 
ceived frame based upon said temporal predic- 
tion produced from said reduced resolution digital 
signal; and 

coding a second digital signal related to 46 
the picture elements off said received frame, 
adaptively employing an estimate based upon 
said temporal and said spatial predictions of said 
received frame. 

60 

7, Apparatus for encoding digital video signals, 
comprising: 

means for receiving a digital video input 
signal including a succession of digital represerv 
tations related to picture elements of at least two 55 
frames of a video image, each of said frames 
comprising a plurality of interlaced odd and even 
fields; 

21 



means for coding a first reduced resolution 
digital signal related to the picture elements of an 
odd field of a received frame, employing. If said 
received frame is not the initial frame received, a 
prediction of said odd f ield of said received frame 
based upon a previously coded odd field from a 
previously received frame; 

means for producing a temporal prediction 
of said odd field of said received irame from said 
first reduced resolution digital signal; 

means for coding a second reduced reso- 
lution digital signal related to the picture elements 
of an even f ield of said received frame, employ- 
ing, if said received frame Is not the initial frame 
received, a prediction of said even field of said re- 
ceived frame based upon a previously coded 
even field from a previously received frame; 

means for producing a temporal prediction 
of said even field of said received frame from said 
second reduced resolution digital signal; 

means for producing a spatial prediction of 
said received frame based upon saM temporal 
predictions produced from said first and second 
reduced resolution digital signals; 

means for producing a temporal prediction 
of said received frame; and 

means for coding a third digital signal re- 
lated to the picture elements of said recewod 
frame, adaptively employing an estimate based 
upon said temporal and said spatial predictions of 
said received frame. 

a. Method for encoding digital video signals, com- 
prising: 

receiving a digital video input signal includ- 
ing a succession of digital representations related 
to picture elements of at least two franr>es of a vid- 
eo image, each of said frames comprising a plur- 
ality of interlaced odd and even fields; 

coding a first reduced resolution digital 
signal related to the picture elements of an odd 
field of a received frame, employing. If said re- 
ceived frame is not the initial frame received, a 
prediction of said odd field of said received frame 
based upon a previously coded odd field from a 
previously received frame; 

producing a temporal prediction of said 
odd field of said received frame from said first re- 
duced resolution digital signal; 

coding a second reduced resolution digital 
signal related to the picture elements of an even 
field of said received frame, employing, if said re- 
ceived frame is not the initial frame received, a 
prediction of said even field of said received 
frame based upon a previously coded even field 
from a previously received frame; 

producing a temporal prediction of said 
even field of said received frame from said see- 
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ond reduced resolution digital signal; 

producing a spatial prediction of said re- 
ceived frame based upon said temporal predic- 
tions produced from said first and second re- 
duced resolution digital signals; 6 

producing a temporal prediction of said re- 
ceived frame; and 

coding a third digital signal related to the 
picture elements of said received frame, adap- 
tively employing an estimate based upon said io 
temporal and said spatial predictions of said re- 
ceived frame. 

9. Apparatus for decoding digital video signals, 
comprising: is 

means for receiving a first and second dig- 
ital signals, said first digital signal representing a 
reduced resolution depiction of a first video inv 
age, and said second digital signal representing 
a full resolution depiction of said first video image; 20 

means for decoding from said first digital 
signal said reduced resolution depiction of said 

first video image, employing in said decoding. If 

said first video image is not the initial image for 
which an input signal was received, a prediction 25 
of said first video image based upon a previously 
decoded video image from a previously received 
input signal; 

means for producing a temporal prediction 
of said first video image from said decoded re- 30 
duced resolution sigrial; 

means for producing a spatial prediction of 
said received first video image based upon said 
reduced resolution video field; and 

means for decoding from said second dig- as 
ital signal a full resolution depiction of said first 
video image, adapted to determine if an estimate 
based upon said temporal prediction of said first 
video image, or an estimate based upon said spa- 
tial prediction of said first video image will be em- 40 
ployed in the decoding of said second digital sig- 
nal. 

10. Method for decoding digital video signals, com- 
prising the steps of: 45 

receiving a first and second digital signals, 
said first digital signal representing a reduced re- 
solution depiction of a first video image, and said 
second digital signal representing a full resolution 
depiction of said first video image; so 

decoding from said first digital signal said 
reduced resolution depiction of said first video irrv 
age, employing in said decoding, if said first video 
image is not the initial image for which an input 
signal was received, a prediction of said first vid- 55 
eo Image based upon a previously decoded video 
image from a previously received input signal; 

producing a temporal prediction of said 



f irst video image from said decoded reduced re- 
solution signal; 

producing a spatial prediction of said re- 
ceived f irst video image based upon said reduced 
resolution video field; and 

decoding from said second digital signal a 
full resolution depiction of said first video image, 
adapted to determine if an estimate based upon 
said temporal prediction of said first video image, 
or an estimate based upon said spatial prediction 
of said first video image will be employed in the 
decoding of said second digital signal. 

11. Apparatus for decoding digital video signals, 
comprising: 

means for receiving a first and second dig- 
ital signals, said first digital signal representing a 
reduced resolution field of a first franr^e of video, 
and said second digital signal representing a full 
resolution field of said first frame of video; 

means for decoding from said first digital 
signal said reduced resolution field, employing In 
said decoding," If said decoded reduced resolution 
field is not the initial field to t>e decoded by said 
apparatus, a prediction of said reduced resolution 
field based upon a previously decoded reduced 
resolution field from a previous frame; 

means for producing a temporal prediction 
of said field of said first frame of video from said 
decoded reduced resolution field; 

means for producing a spatial prediction of 
said video field of said first frame of video based 
upon said reduced resolution video field; and 

means for decoding from said second dig- 
ital signal said full resolution video field, adapted 
to determine if an estimate based upon said tenrv 
poral prediction of said field of said first frame of 
video, or an estimate based upon said spatial pre- 
diction of said field of said first frame of video will 
be employed in the decoding of said second dig- 
ital signal. 

12. Method for decoding digital video signals, com- 
prising the steps of: 

receiving a first and second digital signals, 
said first digital signal representing a reduced re- 
solution field of a first frame of video, and said 
second digital signal representing a full resolution 
field of said first frame of video; 

decoding from said first digital signal said 
reduced resolution field, employing in said de- 
coding, if said decoded reduced resolution field is 
not the initial field to be decoded by said appara- 
tus, a prediction of said reduced resolution field 
based upon a previously decoded reduced reso- 
lution field from a previous frame; 

producing a temporal prediction of said 
field of said first frame of video from said decoded 
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reduced resolution field; 

producing a spatial prediction of said re- 
ceived first field of said first frame of video based 
upon said reduced resolution video field; and 

decoding from said second digital signal 
said full resolution video field, adapted to deter- 
mine if an estimate based upon said temporal pre- 
diction of said field of said first frame of video, or 
an estimate based upon said spatial prediction of 
said field of said first frame of video will be enrv 
ployed in the decoding of said second digital sig- 
nal. 

13. Apparatus for decoding digital video signals, 
comprising: 

means for receiving a first and second dig- 
ital signals, said first digital signal representing a 
reduced resolution frame of a first frame of video, 
and said second digital signal representing a full 
resolution frame of said first frame of video; 

means for decoding from said first digital 
signal said reduced resolution frame, employing 
in said decoding. If said decoded reduced resolu- 
tion frame is not the ihitiarffame to be decoded 
by said apparatus, a prediction of said reduced 
resolution frame based upon a previously decod- 
ed reduced resolution even frame from a previous 
frame; 

means for producing a temporal prediction 
of said first frame from said decoded reduced re- 
solution field; 

means for producing a spatial prediction of 
said first frame based upon said decoded re- 
duced resolution frame; and 

means for decodir^ from said second dig- 
ital signal said full resolution frame of said first 
frame, adapted to determine if an estimate based 
upon said temporal prediction of said first frame, 
or an estimate based upon said spatial prediction 
of said first frame will be employed in the decod- 
ing of said second digital signal 

14. Method for decoding digital video signals, com- 
prising the steps of: 

receiving a first and second digital signals, 
said first digital signal representing a reduced re- 
solution frame of a first frame of video, and said 
second digital signal representing a full resolution 
frame of said first frame of video; 

decoding from said first digital signal said 
reduced resolution frame, employing in said de- 
coding, if said decoded reduced resolution frame 
is not the initial frame to be decoded by said ap- 
paratus, a prediction of said reduced resolution 
frame based upon a previously decoded reduced 
resolution even frame from a previous frame; 

producing a temporal prediction of said 
first frame from said decoded reduced resolution 



field; 

producing a spatial prediction of said first 
frame based upon said decoded reduced resolu- 
tion frame; and 

6 decoding from said second digital signal 

said full resolutbh frame of said first frame, 
adapted to determine K an estinr^te based upon 
said temporal prediction of said first frame, or an 
estimate based upon said spatial prediction of 

10 said first frame will be employed in the decoding 

of said second digital signal. 

15. Apparatus for decoding digital video signals, 
comprising: 

16 means for receiving a first, second, and 

third digital signals, said f irst digital signal repre- 
senting a reduced resolution frame of an odd field 
of a first frame of video, said second digital signal 
representing a reduced resolution frame of an 
20 even field of said first frame of video, and said 

third digital signal representing a lUD resolution 
frame of said first frame of video; 

means for decoding from said first digital 
signal said reduced resolution odd field, employ- 
25 ing In said decoding, if said first frame is riot the 

initial frame being decoded t>y said apparatus, a 
prediction of said reduced resolution odd field 
based upon a previously decoded reduced reso- 
lution odd field from a previous frame; 
30 means for producing a temporal prediction 

of said odd field of said first frame from said de- 
coded reduced resolution odd field; 

means for decoding from said second dig- 
ital signal said reduced resolution even field, em- 
36 ploying in said decoding, if said first frame Is not 

the initial frame being decoded by said appara-- 
tus, a prediction of said reduced resolution even 
field based upon a previously decoded reduced 
resolution even field from a previous fran^; 
40 means for producing a temporal prediction 

of said even field of said first franr>e from said de- 
coded reduced resolution even field; 

means for producing a spatial prediction of 
said first frame based upon said temporal predio- 
46 tions produced from said first and second re- 

duced resolution digital signals; arKl 

means for decoding from said third digital 
signal said full resolution frame of said first frame, 
adapted to determine If an estimate based upon 
50 said temporal prediction of said first frame, or an 

estimate based upon said spatial prediction of 
said first frame will be employed in the decoding 
of said second digital signal. 

55 16. Method for decoding digital video signals, com- 
prising: 

receiving a first, second and third digital 
signals, said first digital signal representing a re- 
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duced resolution frame of an odd field of a first 
frame of video, said second digital signal repre- 
senting a reduced resolution frame of an even 
field of said first frame of video, and said third dig- 
ital signal representing a full resolution frame of 6 
said first frame of video; 

decoding from said first digital signal said 
reduced resolution odd field, employing in said 
decoding, if said first frame is not the initial frame 
being decoded by said apparatus, a prediction of io 
said reduced resolution odd field based upon a 
previously decoded reduced resolution odd field 
from a previous franr>e; 

producing a temporal prediction of said 
odd field of said first frame from said decoded re- is 
duced resolution odd field; 

decoding from said second digital signal 
said reduced resolution even field, employing in 
said decoding, if said first frame is not the initial 
frame being decoded by said apparatus, a predic- 20 
tion of said reduced resolution even field based 
upon a previously decoded reduced resolution 
— even field from a previous^rame;" " 

producing a temporal prediction of said 
even field of said first frame from said decoded 25 
reduced resolution even field; 

producing a spatial prediction of said first 
frame based upon said temporal predictions pro- 
duced from said first and second reduced resolu- 
tion digital signals; and 30 

decoding from said third digital signal said 
full resolution frame of said first frame, adapted to 
determine If an estimate based upon said tempo- 
ral prediction of said first frame, or an estimate 
based upon said spatial prediction of said first 36 
frame will be employed in the decoding of said 
second digital signal. 

17. Apparatus as claimed in claim 1, 3, 5,7, 9, 11, 13 

or 15, or a method as claimed in claim 2, 4, 6, 8, 40 
10, 12, 14 or 16, wherein said reduced resolution 
digital signal is of a format which is compatit>le 
with Motion Picture Experts Group Phase 1 stan- 
dards. 
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